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Introduction 


The  Military  Interoperable  Digital  Hospital  Testbed  (MIDHT)  (Contract  #W81XWH-10-2-0180)  was  a 
program  of  research  to  develop  a  real-world  testbed  environment  in  Southwestern  Pennsylvania.  The 
purpose  was  to  research  and  evaluate  Health  Information  Exchange  (HIE)  and  health  information 
technology  (HIT)  and  services  (HITS)  that  made  health  information  readily  available  to  consumers  and 
providers.  Core  technologies  under  investigation  included  pharmacy  robotics  and  bar  code  medication 
administration  (BCMA).  Concerning  information  exchange,  this  ideally  allowed  for  the  secure  transfer  of 
information  between  private  sector  rural  providers,  federal  partners  and  patients.  MIDHT  also  defined 
requirements  and  solutions  to  optimize  healthcare  resources  for  rural  communities  and  identified 
lessons  learned  and  best  practices  that  benefit  both  the  global  MHS  environment  and  stakeholders  in 
the  region.  The  Department  of  Defense  (DoD)  and  Conemaugh  Memorial  Medical  Center  [Conemaugh; 
alternatively  known  as  the  Conemaugh  Health  System  (CHS)],  have  common  requirements  for  HIE, 
connecting  disparate  systems  and  providers  and  enabling  secure  provider-provider  and  provider- 
consumer  e-communications.  Minimal  evidence  is  available  on  what  business,  clinical  and  technical 
solutions  can  be  used  to  overcome  the  lack  of  specialists,  infrastructure  and  geographical  barriers 
associated  with  the  delivery  of  care  in  rural  communities. 

The  following  scopes  of  MIDHT  research  were  performed  over  a  five-year  time  period  (October  2010  - 
September  2015). 

Arm  1.  The  Impact  of  Medication  Dispensing/ Administration  Technology  Within 
a  Rural  Healthcare  System. 

Subtask  1.1  Implement  pharmacy  robotic  technology  and  bar-coded  enabled  medication 
administration  (BCMA)  in  an  acute  hospital  system  setting. 

Subtask  2.1  Research  and  Analyze  the  resulting  technological  impact  on  medication  errors, 
pharmacists'  productivity,  nurse  satisfactions/workflow  and  patient  satisfaction. 

In  order  to  improve  safety  and  efficiency  of  medication  dispensing  and  administration,  a  complementary 
set  of  health  information  technologies  have  been  implemented.  A  centrally  managed  pharmacy  robotics 
system  was  implemented  at  the  tertiary  care  facility,  Memorial  (MMC),  in  2011.  Many  of  the 
medications  ordered  for  use  on  inpatient  clinical  units  are  currently  dispensed  by  the  Robot  Rx®  system. 
Therefore,  bar  coded  medications  are  now  administered  (BCMA)  at  the  bed  side  to  patients  on  many 
clinical  areas  at  all  three  system  hospitals,  accompanied  by  an  electronic  medication  administration 
record  (eMAR).  Research  objectives  focused  on  medication  errors,  provider  workflow,  provider 
satisfaction  and  related  financial  metrics. 


Arm  2.  Health  Information  Exchange  (HIE)  via  the  Nationwide  Health 
Information  Network  (NHIN). 

Subtask  2.1  Deploy  a  limited  production,  NwHIN  standards-based  HIE  focusing  on  the  bi-directional 

exchange  of  electronic  medical  records  between  CHS  and  the  Military  Health  System.  CHS 
information  to  include  data  domains  residing  in  acute  care  and  ambulatory  settings. 
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Subtask  2.2  Provide  technical  and  documentation  assistance  on  DoD-managed  Virtual  Lifetime 
Electronic  Record  (VLER)  efforts. 

Subtask  2.3  Investigate  productizing  a  Patient  Consent  module  using  established  standards,  such  as 
TP20/XACML. 

Subtask  2.4  Assess  and  analyze  NwHIN-related  activities,  to  include  data  center  performance  metrics, 
physician  evaluation  and  usage  of  the  NHIN  Portal,  and  resulting  benefits  of  HIE  with 
federal  participants. 

NHIN,  developed  by  the  Office  of  the  National  Coordinator  for  Health  Information  Technology  (ONCHIT), 
provides  a  mechanism  for  previously  disconnected  systems  and  exchanges  to  connect  to  each  other  and 
share  health  information  using  nationally  recognized  interoperability  standards,  specifications, 
participation  agreements  and  policies.  The  ability  to  share  data  electronically  among  organizations  will 
provide  a  wide  range  of  benefits  to  citizens,  among  them:  having  up-to-date  records  available  at  the 
point  of  care;  enhancing  population  health  screening;  and  being  able  to  collect  case  research  faster  to 
facilitate  disability  claims.  Rather  than  individually  build  the  software  required  to  make  this  possible, 
the  federal  agencies  collaborated  through  the  Federal  Health  Architecture  (FHA)  program  to  created 
CONNECT,  a  single  open  source  solution  that  can  be  reused  by  each  agency  within  its  own  environment. 
MIDHT  objectives  will  focus  on  executing  a  limited  production  HIE  portal  (using)  NHIN  (standards)  for 
the  purpose  of  standards-based  exchange  of  Military  Health  System  data  domains  with  private  sector 
partners  (e.g.  Conemaugh).  Live  patient  data  will  be  utilized  after  a  NHIN-production  Data  Use 
Agreement  and  Reciprocal  Services  Agreement  (DURSA)  have  been  executed. 

All  abbreviations  and  acronyms  used  throughout  the  document  are  listed  in  Appendix  A. 
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Body 


Arm  1.  The  Impact  of  Medication  Dispensing/ Administration  Technology  Within 
a  Rural  Healthcare  System. 

Subtask  1.1  Implement  pharmacy  robotic  technology  and  bar-coded  enabled  medication 
administration  (BCMA)  in  an  acute  hospital  system  setting. 

Subtask  2.1  Research  and  Analyze  the  resulting  technological  impact  on  medication  errors, 
pharmacists  productivity,  nurse  satisfactions/workflow  and  patient  satisfaction. 

1.1.1  Pharmacy  Robotics  Implementation 

September  30,  2010  -  executed  contract  with  McKesson  Information  Systems  for  Pharmacy  Automation 
technology,  consisting  mainly  of: 

1.  RX  Robot  -  automated  bar-code  driven  medication  management  and  dispensing  system  that  can 
accommodate  up  to  600  different  medications. 

2.  MedCarousel  -  automated  medication  picking  station  that  manages  medications  not  contained 
in  RX  Robot. 

3.  PACMED  -  packaging  system  that  supplies  RX  Robot  and  MedCarousel  with  bar-coded  unit  dose 
medications. 

4.  ConnectRX  database  server  -  single  database  server  that  supports  integration  of  RX  Robot, 
MedCarousel,  and  PacMed  technologies  and  provides  single  point  of  interface  into  overall 
Conemaugh  Health  System  (CHS)  network  environment. 

November  12,  2010  -  Preliminary  facility  plans  provided  by  McKesson  were  presented  to  the 
Pennsylvania  Department  of  Health  Safety  Division  in  Harrisburg.  During  the  review,  the  Department 
determined  that  a  full  review  of  complete  plans  were  required  before  any  renovation  work  could  begin. 
They  based  this  decision  on  the  scope  of  structural  reinforcement  changes  needed  to  accommodate  the 
weight  of  the  RX  automation  equipment  in  the  MMC  Pharmacy  department.  We  submitted  complete 
structural,  electrical,  power  and  cooling  engineering  drawings  to  the  Department  on  January  14,  2011. 

Conemaugh  contracted  with  H.F.  Lenz  Engineering  Group  to  create  engineering  drawings  of  facility 
renovations  necessary  for  installation  of  Robot-Rx,  MedCarousel,  and  PACMED. 

December  20,  2010  -  Conducted  a  conference  call  with  McKesson,  Conemaugh,  and  Care  Fusion  to 
review  technical  requirements  of  a  new  HL7  interface  between  the  McKesson's  ConnectRX  platform  that 
serves  the  RX  automation  equipment  and  the  Pyxis  dispensing  cabinet  systems  deployed  throughout 
CHS  nurse  stations.  This  interface  enabled  Pyxis  cabinets  to  automatically  notify  MedCarousel 
dispensing  system  (via  ConnectRX  interface)  when  medication  restockings  are  required.  This  degree  of 
automation  improved  efficiency  of  Pharmacy  operations. 

The  following  tasks  were  completed  in  January  -  March  2011: 

•  Racked  and  staged  connected  to  the  network  in  our  data  center  the  Connect-Rx  server,  which 
controls  Pharmacy  automation  components  including  the  Robot-Rx,  MedCarousel  and  PACMED. 
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Racked  and  staged  connected  to  the  network  the  Connect-Rx  test  server  in  our  data  center, 
which  was  used  for  testing  purposes. 

•  Racked  and  staged  connected  to  the  network  in  our  data  center  the  Interface  server,  which 
provides  interface  capabilities  between  the  pharmacy  order  system  and  pharmacy  automation 
systems. 

•  Test  workstation  in  Pharmacy  deployed  which  will  allow  for  testing  of  the  automation  products. 
Support  workstation  deployed  for  Connect-Rx  in  pharmacy,  which  allows  for  support  of  the 
Connect-Rx  server. 

•  All  pharmacy  renovation  approvals  obtained  at  state  and  local  level.  Electrical  requirements 
finalized  and  installed,  including  UPS  and  emergency  power.  In  addition,  floor  re-enforcement 
and  compressed  air  system  for  the  Robot  has  been  completed. 

•  Pyxis  interface  installed  (this  is  required  for  Rx  automation  to  communicate  with  existing 
automated  medication  dispensing  cabinets  located  on  nurse  stations) 

•  National  Drug  Codes  (NDC)  defined  in  pharmacy  information  system 

•  Existing  pneumatic  tube  system  re-directed  to  allow  for  MedCarousel  placement 

•  Data  cabling  for  equipment  and  workstations  completed 

The  following  tasks  were  completed  in  April  -  June  2011: 

•  Facility  Prep  (compressed  air,  UPS  power)  -  completed  April  11,  2011 

•  MedCarousel  -  installed  April  11,  2011 

•  Robot/Overwrap  Packager  -  installed  April  18,  2011 

•  PACMED/Singulator  -  installed  April  18,  2011 

•  Pyxis  ADC  Interface  -  installed  April  1,  2011 

•  Data  cabling  -  completed  April  19,  2011 

•  First  Data  Bank  Integration  into  ConnectRX  -  completed  April  14,  2011 

•  MedCarousel  Core  Team  attended  training  in  Cranberry,  PA:  April  6-8,  2011 

•  PACMED  Core  Team  attended  training  in  Cranberry,  PA:  April  26-27,  2011 

•  Robot  Core  Team  attended  training  in  Cranberry,  PA:  April  28-29,  2011 

On  Site  Training: 

MedCarousel  Core  Team  Training  -  April  19-20,  2011 
PACMED  /  Robot  Core  Team  Training  -  April  26-29,  2011 
PACMED  User  Support  -  May  3-5,  2011 
Robot  Inventory  Prep  -  May  9-11,  2011 
Inventory  Prep  Training  -  May  10-12,  2011 

Acceptance  Demonstration: 

MedCarousel  -  April  28,  2011 
Robot  -  May  3,  2011 
PACMED -May  3,  2011 

During  the  last  quarter  of  2011,  the  robotic  automation  effort  was  fully  deployed  for  MMC  and 
Conemaugh  Meyersdale  Medical  Center  (MYMC).  Fulfill-Rx  automated  re-ordering  training  and 
operational  status  went  live  on  August  30,  2011.  Numerous  equipment-related,  workflow-related,  and 
training-related  issues  came  to  light  as  more  nurse  stations  came  on  board  with  the  new  process. 
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Equipment  issues  were  particularly  troublesome  and  created  challenges  for  the  pharmacy  staff.  These 
issues  included  the  filling  speed  and  location  of  stat  orders,  repackager  inconsistencies  and  applicability 
to  certain  medications,  suction  issues  with  the  robotic  arm  that  resulted  in  dropped  (missing) 
medications,  and  increased  noise  levels.  Weekly  support  calls  were  held  and  several  field  adjustments 
were  made  by  the  vendor  on  both  the  Robot  and  MedCarousel. 

Performance  and  uptime  improved  incrementally  during  the  period.  By  September  27,  2011,  all  but  a 
handful  of  issues  remained  outstanding.  One  area  of  continued  concern  was  the  envelope  delivery 
system,  or  EDS.  This  is  the  mechanism  whereby  the  Robot  places  picked  meds  into  patient-specific 
cardstock  envelopes  for  transportation  to  the  nursing  station.  The  EDS  will  sporadically  malfunction  and 
cause  some  medications  to  miss  the  envelope.  The  vendor  has  improved  the  process  via  equipment 
adjustments,  but  reliability  issues  remained. 

Unanticipated  workflow  issues  for  nursing  came  to  light  as  the  new  process  was  rolled  out.  One  in 
particular  was  the  continuation  of  previous  habits.  The  previous  decentralized  medication  distribution 
model  required  nursing  to  pull  all  patient  medications  from  automated  dispensing  cabinets  (ADCs) 
located  on  each  nurse  station.  During  the  project  planning  phase,  pharmacy  decided  to  leave  the 
medications  in  the  ADCs  as  a  precaution  in  the  event  the  new  centralized  distribution  process  had 
problems.  Pharmacy  discovered  that  nursing  was  continuing  to  pull  all  their  medications  from  the  ADC 
instead  of  from  the  envelopes.  The  correct  process  is  to  pull  only  narcotics  from  the  ADC,  since  all  other 
medications  are  to  be  in  the  envelope.  This  resulted  in  envelopes  being  returned  to  the  pharmacy  full  of 
medications  and  requiring  manual  crediting  to  the  patient's  account.  The  issue  was  exacerbated  by  the 
fact  that  the  bar  code  medication  administration  (BCMA)  process  was  not  yet  deployed  and  nursing  did 
not  have  the  new  carts  in  place  to  house  the  medications  sent  from  the  pharmacy.  Pharmacy  addressed 
the  issue  by  providing  additional  education  to  nursing  and  by  removing  the  scheduled  meds  from  the 
ADCs  so  that  nursing  must  use  the  envelopes.  Anecdotal  concerns  exist  from  nursing  as  they  need  to 
empty  envelopes  before  placing  specific  medications  into  drawers  on  the  mobile  medication  carts. 

A  second  workflow  issue  involved  hours  of  pharmacy  operation.  Under  the  decentralized  model,  night- 
shift  was  a  slow  time  with  minimal  activity  other  than  off-hours  order  processing.  This  changed  in  the 
new  Robot-driven  centralized  workflow  model.  Pharmacy  learned  that  they  need  to  utilize  the  midnight 
shift  to  perform  cart  fill  so  that  the  envelopes  for  the  next  day  can  be  delivered  to  Nursing  in  the  early 
morning.  This  change  required  a  review  of  staffing  so  that  additional  staff  is  scheduled  for  the  midnight 
shift. 

A  final  operational  issue  involved  the  capacity  of  the  automation  system.  We  have  learned  that,  even 
with  optimization  efforts,  the  combined  capacity  of  the  Robot  and  MedCarousel  are  not  sufficient  to 
meet  the  needs  of  Conemaugh  operations.  A  re-budget  request  is  currently  being  developed  to  address 
this  issue  by  adding  a  second  MedCarousel.  Project  team  identified  placement  location  for  second 
MedCarousel  cabinet  in  Pharmacy.  Configuration  was  finalized  and  facility  drawings  were  drafted.  As 
was  required  for  the  initial  Pharmacy  robotics  installation,  the  facility  changes  required  to  support  the 
second  MedCarousel  required  Pennsylvania  Department  of  Health  approval  before  work  could  begin. 
Conemaugh  contracted  with  McKesson  on  September  26,  2012  for  the  second  MedCarousel  dispensing 
system. 

Conemaugh  finalized  the  second  MedCarousel  dispensing  system  placement  in  the  centralized  MMC 
pharmacy.  Conemaugh  worked  with  an  outside  engineering  firm  to  prepare  documents  that  were 
submitted  to  the  Pennsylvania  Department  of  Health  in  January  2013  for  approval  before  physical 
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accommodation  work  began.  The  amount  of  site  work  required  was  much  less  than  with  first 
MedCarousel  due  to  the  fact  that  the  second  unit  was  not  as  tall  as  the  first  one  and  would  not  protrude 
through  the  ceiling.  Floor  support  was  the  primary  required  change. 

The  second  MedCarousel  implementation  was  implemented  in  the  pharmacy  on  April  8,  2013. 
Conemaugh  streamlined  the  manual  pick  processing  that  occurs  when  a  medication  is  not  able  to  be 
stocked  in  the  robot.  This  also  has  helped  reduce  the  number  of  open  bulk  medication  bottles  and 
decrease  inventory  slightly  because  of  increased  efficiency.  The  second  MedCarousel  has  also  helped 
with  the  picking  processes  since  many  of  the  oral  medications  are  located  in  the  MedCarousel.  The 
barcode  system  is  fully  engaged  and  ensures  patient  safety  each  time  a  medication  is  removed  from  the 
MedCarousel.  Technician  time  is  better  spent  doing  other  tasks  than  searching  for  a  missing  medication. 
The  second  MedCarousel  has  helped  Conemaugh  achieve  a  better  work  flow  pattern  in  the  drug 
distribution  area.  The  additional  MedCarousel  has  also  helped  with  the  ordering  and  restocking  of 
inventory  when  it  arrives  within  the  department  from  the  manufacturer.  With  barcode  scanning, 
Conemaugh  can  ensure  that  the  medication  is  in  its  proper  place  when  being  restocked.  As  with  the  first 
MedCarousel,  PAR  levels  were  setup  to  help  keep  inventory  levels  sufficient  to  meet  patient  needs. 


Figure  1.  Robot-Rx  in  MMC  Pharmacy. 
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Figure  1  illustrates  the  MMC  pharmacy  Robot-RX.  Please  see  Appendix  B  for  additional  pictures  of  the 
pharmacy  robotics  equipment. 


1.1.2  Bar  Code  Medication  Administration  (BCMA)  "Horizon  Admin  Rx"  Implementation 

After  execution  of  the  contract  with  McKesson  for  the  Horizon  Admin  Rx  project  in  late  2010,  the 
following  tasks  were  completed: 

•  Vendor  and  internal  project  managers  assigned 

•  Nursing  champions  identified  to  assist  in  workflow  design 

•  AdminRx  project  kick-off  conference  call  with  vendor  conducted  on  March  14,  2011 

•  Build  training  of  MIS  and  pharmacy  staff 

•  Wireless  infrastructure  assessed,  recommendations  made 

•  MIS  and  pharmacy  staff  completed  Admin  Rx  on-line  education 

•  Completed  Admin  Rx  System  build 

•  Carelink  interface  migration  for  training  and  QA  environments 

•  McKesson  Implementation  Team  visit  to  begin  bar  code  verification 

•  Citrix  environment  testing 

•  Development  of  user  training  material 

•  Training  schedules  completed 

•  Set-up  and  tested  Admin  Rx  reports  to  Horizon  Patient  Folder 

•  Table  copied  from  facility  01  to  facility  02 

•  Continued  bar  code  verification 

•  Device  selection  finalized  and  initial  equipment  order  initiated 

•  End  user  training  completed  -  4  hour  classroom  session  with  hands-on  practice 

•  Training  of  Super  User  Support  Team,  which  consists  of  10  nurses  loaned  to  MIS  by  nursing  to 
assist  in  the  Admin  Rx  rollout.  These  10  nurses  received  40  hours  of  classroom  training  by  MIS 
project  team  experts. 

•  Device  configuration  and  testing 


The  first  Pilot  units  consisted  of  Ashman  7,  Rose  7  and  Cardiac  Intensive  Care  Unit  (CICU)  and  they  went 
live  on  September  27,  2011. 

After  the  pilot  live  deployment  was  successful,  the  organization  completed  the  following  phased  Roll-out 
Schedule  in  the  Fall  of  2011  and  Winter  of  2012.  Devices  (Rubbermaid  carts,  wireless  scanners)  were 
deployed  to  nursing  units  according  to  the  live  schedule  below.  End  user  classroom  Admin  Rx  training 
was  provided  frequently  for  nursing  units.  IT  staff  and  nursing  Super  Users  provided  24x7  live  support  to 
nursing  units  as  needed. 

October  2011 

Ashman  8th,  9th  and  10th  clinical  units 
Rose  8th,  9th  and  10th  clinical  units 
Main  7,  Pediatrics  and  Women  Services 

November  2011 

Good  Sam  4th,  5th  and  6th  clinical  units 
Behavioral  Medicine  Good  Sam  7th  clinical  unit 


10 


December  2011 

Geropsych  Good  Sam  8th  clinical  unit 
Aloysia  Hall  Good  Sam  8th  clinical  unit 
E  Building  4th  and  6th  clinical  units 

January  -  March  2012 

Crichton  Center 

Intensive  Care  Unit  -  Ashman  6 

Intensive  Care  Unit  -  Rose  6 


Figure  2.  Rubbermaid  Medication  Cart. 

BCMA  adoption  rate  for  January  2012  was  88%  of  total  medications  administered  for  inpatients  at  MMC, 
87%  of  February  and  86%  for  March.  Adoption  rate  is  defined  as  the  number  of  medication 
administrations  where  all  aspects  were  completed  via  bar-coding  (wristband  check,  medication  barcode, 
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wristband  verification)  divided  by  the  total  number  of  medication  administrations  for  the  same  period. 
All  inpatient  units  at  MMC  were  live  as  of  the  end  of  January  2012. 

Pharmacy  and  IT  staff  began  meetings  with  MYMC  regarding  Admin  Rx  implementation  at  their  facility. 
McKesson  Domain  Expertise  Group  performed  the  Post-Live  Optimization  Assessment  at  MMC.  MYMC 
build  and  planning  continued,  including  device  assessment  and  purchasing. 

The  Post-live  optimization  assessment  was  shared  with  the  Clinical  Steering  Committee.  Nursing, 

Patient  Access,  Health  Information  Systems,  and  IT  discussed  the  need  for  bar-code  labels  to  be  printed 
on  nursing  units  to  facilitate  BCMA.  Testing  of  build  for  the  MYMC  facility  and  staff  training  were 
started. 

MYMC  went  live  with  BCMA  on  April  17,  2012  as  planned  and  achieved  a  closed-loop  BCMA  adoption 
rate  of  95%  in  the  first  month.  This  rate  was  higher  than  any  department  at  MMC  at  the  current  time. 

The  patient  ID  label  printer  Pilot  with  Ashman  6  Nurse  Station  was  completed  in  late  April  and  began  full 
deployment  in  May.  All  but  two  of  the  22  Nurse  Station  at  MMC  was  equipped  with  ID  label  printers  as 
of  June  20,  2012.  With  these  printers  in  place.  Nursing  could  now  locally  print  replacement  bar-code 
wristband  labels  when  needed.  Feedback  from  Nurse  Managers  was  very  positive.  The  printers  have 
been  so  popular  with  Nursing  that  we  are  also  now  using  these  same  printers  to  print  "form  labels". 
Form  labels  provide  improved  legibility  vs.  imprinting  the  forms  with  a  physical  ID  card  and 
Addressograph  machine.  The  IT  department  successfully  implemented  the  vl0.3  version  upgrade  of 
AdminRX  application  and  associated  Care  Organizer  application  on  June  26,  2012. 

IT  staff  collaborated  with  the  Respiratory  Department  in  an  effort  to  address  staff  complaints  regarding 
difficulty  pushing  the  Rubbermaid  medication  carts.  In  contrast  to  Nursing,  workflow  at  Respiratory 
requires  them  to  move  the  carts  longer  distances  and  also  onto  elevators.  Beginning  May  7,  2012,  a 
differently  configured  cart  consisting  of  a  lithium  battery  and  larger  cart  wheels  was  put  into  Pilot  use. 

The  organization  increased  the  Conemaugh  BCMA  adoption  rate  to  96%  for  September  2012  from  88% 
in  June  and  86%  in  March.  The  significant  improvement  was  achieved  primarily  by  providing  nurse 
managers  with  staff-specific  performance  reports.  Nurse  Managers  used  these  reports  to  work  with 
staff  whose  rates  were  low  to  determine  the  reasons  and  address  them.  MYMC  maintained  their 
adoption  rate  at  a  consistent  94%  -  96%  range  through  September. 

Conemaugh  contracted  with  McKesson  for  the  Medication  Safety  Analytics  application.  McKesson 
installed  the  BCMA  Analytics  feed  into  the  performance  management  system  called  Horizon  Business 
Intelligence  (HBI).  The  Decision  Support  department  conducted  user  training  for  nurse  managers  and 
directors  on  the  use  of  HBI  to  interact  directly  with  the  BCMA  data.  This  is  a  very  rich  analytics  tool;  and 
once  trained,  nurse  leaders  were  able  to  drill  into  the  data  to  identify  specific  staff,  nurse  stations,  and 
medications  that  require  follow-up  and  possible  re-training.  Screenshots  located  in  Appendix  C  provide 
a  look  and  feel  of  the  BCMA  highlights  via  HBI  and  the  different  customizable  view  levels  (by  unit,  by 
med,  by  staff,  etc.).  Unfortunately  only  three  nurse  managers  (Critical  Care,  OB/GYN  and  Med  -  Surg) 
took  the  initiative  (as  of  February  19,  2015)  to  retrieve  the  data  directly  from  HBI  for  a  grand  total  of  five 
accesses.  Tools  and  information  are  only  productive  if  used  by  people  who  can  impact  change. 

IT  staff  worked  with  leadership  of  the  School  of  Nursing  on  device  needs  for  student  nurses.  Issues 
included  defining  locations  to  house  the  carts  when  not  in  use  by  the  students  and  whether  or  not  these 
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carts  are  to  be  reserved  for  the  exclusive  use  by  students.  Cart  storage  is  a  significant  issue  at 
Conemaugh  where  nurse  stations  are  already  overly  congested.  In  addition,  we  are  under  constant 
pressure  from  the  State  of  Pennsylvania  to  keep  hallways  clear.  Conemaugh  staff  finalized  the  School  of 
Nursing  device  needs  for  student  nurses  in  November  2012.  Additional  Rubbermaid  medication  carts 
were  deployed  in  December  2012  after  receipt  from  vendor.  The  primary  purpose  was  to  improve 
student  exposure  to  the  current  BCMA  process  and  enhance  the  learning  experience. 

Under  the  current  process,  paper  envelopes  are  delivered  to  nursing  units  each  day.  Nursing  then  must 
transfer  the  medications  from  the  envelopes  into  the  plastic  bins  housed  in  the  computerized  med  carts. 
Unused  medications  are  returned  back  to  the  pharmacy  in  the  same  envelopes.  This  process  has 
created  extra  steps  for  nursing  and  thus  has  been  identified  as  an  area  of  dissatisfaction.  Since  the 
envelopes  are  re-used  as  a  cost  saving  strategy,  the  extra  handling  associated  with  this  process  leads  to 
the  envelopes  becoming  damaged.  This,  in  turn,  contributes  to  dispensing  malfunctions  from  the  robot. 
The  Pharmacy  department  has  since  worked  closely  with  nursing  to  pilot  a  new  "exchange  bin"  method 
of  delivering  medications  to  inpatient  units.  In  this  workflow,  nursing  gets  patient-identified  plastic  bins 
with  the  robot-filled  medications  from  the  current  day.  These  bins  are  exchanged  with  bins  from  the 
preceding  day  for  the  same  patient  in  the  med  cart.  The  expected  advantages  were:  1)  Envelopes  never 
leave  the  pharmacy  so  they  are  much  less  likely  to  get  mis-handled  and  damaged  (saves  cost)  and  2) 
Nursing  does  not  have  to  transfer  meds  from  envelope  to  bin  and  then  transfer  unused  meds  from  bin 
to  envelope  (less  chance  for  error).  Feedback  from  the  first  two  pilot  units  was  positive. 

BCMA  implementation  at  Conemaugh  Miners  Medical  Center  (MIMC)  was  delayed.  System-wide 
formulary  consolidation  efforts  were  required  before  BCMA  can  be  implemented  in  a  way  that  can  be 
effectively  supported.  In  May  2013,  MIMC  pharmacy  medication  master  file  was  successfully 
synchronized  with  that  of  MMC  and  MYMC.  A  single  medication  master  file  now  serves  all  three 
Conemaugh  hospitals.  This  was  a  multi-month  effort  that  completed  a  key  prerequisite  to  implementing 
BCMA  at  MIMC.  As  this  task  was  completed,  clinical  analysts  began  the  order  build  process.  Meetings 
were  held  at  the  MIMC  facility  with  nursing  and  ancillary  staff  to  design  the  workflow  process.  MIS 
worked  with  MIMC  Nursing  Administration  to  identify  super  users  to  facilitate  BCMA  training  and  on-site 
support,  similar  to  the  process  used  for  the  MMC  implementation.  MIS  recruited  the  services  of  several 
nursing  super  users  to  dedicate  time  to  these  tasks.  BCMA  deployment  at  MIMC  was  further  delayed 
due  to  implementation  of  the  McKesson  ER12  upgrade.  Design  work  was  started  in  early  November. 
User  training  occurred  in  late  January/early  February,  with  full  LIVE  use  beginning  on  February  25,  2014. 
Adoption  rate  for  the  first  full  month  (March  2014)  exceeded  96%  and  has  remained  strong  since  then. 

The  McKesson  Pharmacy  System  vlO.l  Upgrade  was  successfully  completed  on  October  22,  2013  along 
with  other  clinicals.  The  combination  of  these  upgrades  provided  additional  BCMA  functionality  for 
nurses,  specifically  in  the  area  of  IV  administration.  Benefits  of  the  upgrade  release  were  inclusive  of 
the  BCMA  system  at  MIMC  as  well. 

MMC  pharmacy  and  nursing  finalized  their  medication  exchange  cart  pilots  and  identified  a  workflow 
process  for  hospital-wide  deployment.  The  process  identified  the  pharmacy  as  maintaining  the 
envelopes  and  then  transferring  all  medications  into  exchange  bins  before  delivery  to  nursing  each  day. 
This  approach  optimized  the  robot  usage  by  allowing  the  filling  process  to  remain  unattended.  The 
Medication  Exchange  cart  system  (transfer  carts  and  exchange  bins),  originally  ordered  in  January  2014, 
was  not  deployed  into  clinical  areas  until  August  -  September  2014  due  to  supplier  (Rubbermaid) 
backorder  and  pricing  issues.  Configuration  and  bids  were  more  costly  than  estimated  due  to  larger 
transfer  carts. 
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Figure  3  and  Figure  4  depict  the  bar  code  scanning  rate  for  CHS  by  each  medical  center.  Reasons  for  not 
achieving  a  100%  compliance  rate  can  been  found  in  the  "Lessons  Learned"  section.  The  95%  overall 
average  is  still  very  high  and  supports  adoption  of  the  BCMA  process  by  the  entire  organization.  MYMC 
achieved  the  same  rate  as  MMC  and  MIMC  had  a  higher  rate  than  MIMC.  Staff  at  the  rural  medical 
centers  have  fully  adopted  and  supported  the  BCMA  process. 


Figure  3.  CHS  Medications  Scanned  Using  Barcode. 


Figure  4.  MMC,  MYMC  &  MIMC  Medications  Scanned  Using  Barcode. 
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Subtask  1.2  Research  and  analyze  the  resulting  technological  impact  on  medication  errors, 
pharmacist  productivity,  nurse  satisfaction/workflow  and  patient  satisfaction. 

Both  of  the  following  hypotheses  have  been  rejected: 

1.  We  hypothesize  that  the  implementation  of  a  centralized  pharmacy  robotic  system 
within  the  Conemaugh  Health  System  (CHS)  will  significantly  reduce  medication 
expenses  and  ending  inventory,  increase  pharmacy  productivity  and  reduce  the  number 
of  nonreturnable  expired  medications  and  associated  cost. 

2.  We  hypothesize  that  the  implementation  of  bar  code  medication  administration 
technology  within  the  Conemaugh  Health  System  (CHS)  will  significantly  reduce  the 
amount  of  actual  medication  errors  and  positively  impact  provider  and  patient 
satisfaction. 

1.2.1  MMC  Pharmacist  Productivity 

Data  contained  within  the  Horizon  Meds  Manager  (HMM)  system  could  not  be  collected  in  a  consistent 
and  reliable  manner  across  the  Pre  and  Post  time  periods  due  to  managerial  turnover,  system  upgrades 
and  report  formatting  changes.  The  inconsistent  and  missing  data  therefore  was  not  suitable  for 
analysis  by  the  research  team. 

1.2.2  MMC  Pharmacy  Automation/Robotics  Dispensing  Survey  (Pharmacy  satisfaction) 

Thirty-eight  (38)  pharmacists  and  pharmacy  technicians  completed  the  internally  developed  Pharmacy 
Survey.  Responses  were  collected  from  December  2011  through  January  2012.  The  respondents 
indicated  a  diversity  of  years  of  experience,  with  53%  indicating  greater  than  15  years  of  experience, 

26%  with  5  or  less,  and  the  remaining  21%  from  6-15  years.  A  copy  of  the  survey  is  included  in  Appendix 
D. 

The  majority  of  those  surveyed  responded  favorably.  Summary  points  from  the  survey  are  provided 
below. 

•  Nearly  40%  of  pharmacists  and  pharmacy  technicians  indicated  they  believed  the  McKesson 
Robot-RX®  system  increased  filling  accuracy.  Roughly  30%  were  neutral  however. 

•  92%  (those  who  strongly  disagreed  or  disagreed)  felt  the  system  had  not  reduced  missing 
medications. 

•  The  impact  of  the  system  on  expired  medications  was  nearly  evenly  distributed.  38%  felt 
expired  medications  issues  were  minimized. 

•  Most  (87%)  felt  checking  time  and  the  delivery  of  medications  were  not  reduced. 

•  Nearly  75%  of  respondents  rated  the  Robot-RX®  system  as  difficult  or  very  difficult  to  use,  with 
the  system  affecting  over  70%  of  medications. 
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Q1  From  your  perspective,  the  McKesson 
Robot-RX®  system  has  increased  filling 
accuracy. 

Answered:  38  Skipped:  1 


28.95%  Mil 


Q2  From  your  perspective,  the  McKesson 
Robot-RX®  system  has  significantlyreduced 
missing  medications. 

Answered:  38  Skipped:  1 


Strongly  Disagree 
65.79%  (25) 


Disagree 
26.32%  (10) 


Q3  From  your  perspective,  the  McKesson 
Robot-RX®  system  has  minimized  issues 
with  expired  medications. 

Answered:  37  Skipped:  2 


29.73%  (11) 


Q4  From  your  perspective,  the  McKesson 
Robot-RX®  system  has  significantly 
reduced  the  amount  of  checking  time 
needed  and  medications  are  delivered  to  the 
floor  quicker. 

Answered:  38  Skipped:  1 


Q5  How  would  you  rate  the  Robot-Rx® 
system  and  associated  components  with 
respect  to  ease  of  use. 

Answered:  38  Skipped:  1 


Q6  What  percentage  of  the  hospital’s 
medications  has  been  impacted  by  this 
system? 

Answered:  38  Skipped:  1 


21.05%  (8) 


Figure  5.  Pharmacy  Survey  Questions  1,  2,  3,  4,  5  &  6. 
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In  the  opinion  of  the  pharmacists  and  pharmacy  technicians  surveyed,  there  is  now  less  time  available  to 
spend  on  clinical  activities.  Over  60%  disagreed  (or  strongly  disagreed)  that  patient  safety,  as  reported 
via  a  reduction  of  "near  misses"  or  reportable  was  not  improved.  Eighty  percent  (80%)  would  not 
recommend  the  system  to  other  hospitals  (data  not  shown). 


Figure  6. 


Q7  You  have  been  able  to  spend  more  time 
on  clinical  activities  since  the  new  system 
was  implemented? 

Answered:  38  Skipped:  1 


100% 

80% 

76.32% 

60% 

40% 

23.68% 

20% 

0% 

True 

False 

N/A 

Q9  From  your  perspective,  the  McKesson 
Robot-RX®  system  has  improved  patient 
safety  through  a  reduction  in  “near  misses” 
and  reportable  medication  errors. 

Answered:  38  Skipped:  1 


100% 


80% 


Strongly  Agree  Neutral  Disagree  Strongly 

Agree  Disagree 


Pharmacy  Survey  Questions  7  &  9. 


Additional  statistical  analyses,  chi-square  crosstabs  by  "Would  you  recommend  the  system  (pharmacy 
robotics)  to  other  hospitals?"  and  years  of  service,  were  attempted  and  found  to  be  not  tenable  due  to 
violations  of  one  of  the  underlying  statistical  assumptions,  namely,  the  maximum  percentage  of  cells 
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with  an  expected  frequency  less  than  five.  A  non-parametric  correlation  analyses  was  also  attempted. 
It  too  was  inconclusive  because  of  violation  of  the  monotonic  assumption. 


Medication  distribution  systems  have  changed  significantly  during  the  past  decade.  In  2011,  63%  of 
hospitals  used  automated  dispensing  as  the  primary  method  for  supplying  medications,  while  29%  used 
a  manual  centralized  unit  dose  system.  Such  usage  is  a  reversed  trend  from  only  a  few  years  ago 
(Pedersen  CA  2012).  Despite  the  fact  the  50%  of  hospitals  now  utilize  a  BCMA  system  and  another  11% 
robotic  dispensing,  the  culture  of  change  is  not  always  perceived  as  favorably  despite  known  benefits  to 
both  patient  safety  and  efficiencies.  Pharmacy  staff  at  MMC  viewed  the  implemented  robotics 
dispensing  system  unfavorably,  with  the  majority  noting  the  system  was  difficult  to  use  and  negatively 
impacted  clinical  processes.  Most  failed  to  note  a  positive  impact  on  patient  safety  through  a  reductions 
of  reportable  medication  errors  and  "near  misses." 

1.2.3  Pharmacy  Workflow/Time  and  Motion  Study  -  MMC 

The  research  team  conducted  time  and  motion  observations  of  MMC  pharmacy  both  before  the 
implementation  of  the  BCMA  system  (PRE)  and  then  afterwards  (POST).  A  total  of  12  observations 
were  performed  during  each  time  period.  Varying  shifts  were  selected  to  create  a  representative 
sample.  Observations  occurred  during  continuous  4-hour  periods. 

Data  collection  was  similarly  matched  between  PRE  and  POST  periods  including  the  same 
day(s)/time(s)  of  the  week  and  similar  time  of  year  (seasonality).  Daily  patient  census  was  closely 
matched  as  well  [average  census  (PRE)  =  403,  average  census  (POST)  =  463]. 

PRE  (April  2011) 

12  baseline  observations  were  completed  during  the  month  of  April  2011.  Each  pharmacy  personnel 
were  shadowed  as  detailed  in  Table  1  below. 

As  depicted  below  in  Figure  7,  approximately  27%  of  the  time  was  used  to  fill  medication  orders.  The  next 
four  most  time  consuming  activities  were  medication  order  entry  by  pharmacists  (18%),  walking  in  the 
department  or  throughout  the  hospital  (13%),  loading  medications  into  Pyxis  machines  (9%),  and  talking 
on  the  phone  to  hospital  staff  (8%). 


Date 

Time 

Position 

April  11,  2011 

7:00-11:00  AM 

Pharmacist 

April  12,  2011 

9:00  PM  - 1:00  AM 

Pharmacist 

April  13,  2011 

8:30  AM-12:30  PM 

Technician 

April  14,  2011 

2:30-6:30  PM 

Pharmacist 

April  15,  2011 

8:30  AM -12:30  PM 

Technician 

April  18,  2011 

7:00-11:00  AM 

Technician 

April  18,  2011 

12:00-4:00  PM 

Pharmacist 

April  20,  2011 

12:00-4:00  PM 

Technician 

April  22,  2011 

7:00-11:00  AM 

Technician 

April  25,  2011 

11:00  AM -3:00  PM 

Technician 

April  26,  2011 

9:00  PM -1:00  AM 

Technician 

April  27,  2011 

3:00-7:00  PM 

Technician 

Table  1.  Pharmacy  Time  &  Motion  Schedule  (PRE). 
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Pharmacy  Shadowing:  Summary  of  Time  &  Motion  Analysis,  PRE  period 


□  Total  Time  /  Group 


■  %  of  Total  Time  per  Group 
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Figure  7.  Pharmacy  Shadowing  Time  &  Motion  Summary  (PRE). 

POST  (April  -  May  2013) 

Twelve  time  and  motion  observations  of  pharmacy  personnel  were  again  conducted  post 
implementation  of  the  pharmacy  automation  system,  after  installation  of  the  second  MedCarousel  and 
the  same  time  of  year.  Each  pharmacy  personnel  were  again  shadowed  for  a  4-hour  period  as  detailed 
in  Table  2. 


Date 

Time 

Position 

April  22,  2013 

7:00-11:00  AM 

Pharmacist 

April  23,  2013 

9:00  PM -1:00  AM 

Pharmacist 

April  24,  2013 

8:30  AM -12:30  PM 

Technician 

April  29,  2013 

12:00 -4:00  PM 

Pharmacist 

April  30,  2013 

9:00  PM  - 1:00  AM 

Technician 

May  2,  2013 

2:30-6:30  PM 

Pharmacist 

May  8,  2013 

10:00  AM  -  2:00  PM 

Technician 

May  13,  2013 

11:00  AM -3:00  PM 

Technician 

May  15,  2013 

3:00-7:00  PM 

Technician 

May  17,  2013 

8:30  AM-12:30  PM 

Technician 

May  20,  2013 

7:00-11:00  AM 

Technician 

May  24,  2013 

7:00-11:00  AM 

Technician 

Table  2.  Pharmacy  Time  &  Motion  Schedule  (POST). 
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As  depicted  below  in  Figure  8,  approximately  27%  of  the  time  was  used  to  fill  medication  orders,  which 
was  the  same  amount  of  time  that  was  observed  in  the  PRE  time  period.  The  next  four  most  time 
consuming  activities  were  medication  order  entry  by  pharmacists  (18%),  personal  time  (13%),  loading 
medications  into  Pyxis  machines  (8%),  and  walking  in  the  department  or  throughout  the  hospital  (8%). 


Pharmacy  Shadowing:  Summary  of  Time  &  Motion  Analysis,  POST  period 


Total  Time  /  Group 


co  12:00:00 
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0:00:00  4* 
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Activity  Grouping 
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Figure  8.  Pharmacy  Shadowing  Time  &  Motion  Summary  (POST). 

Data  was  then  compared  between  the  PRE  and  POST  periods.  Figure  9  and  Figure  10  displays 
descriptive  results  per  process  grouping  by  percentage  and  total  time  respectively. 
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Percent  of  Total  Time 


Percent  of  Total  Time  by  Process 

50% 
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Figure  9.  Percent  of  Total  Time  by  Process  (PRE  VS.  POST). 


Total  Time  per  Process 
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Figure  10.  Total  Time  Per  Process  (PRE  VS.  POST). 
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Though  not  necessarily  depicted  in  the  data,  the  infusion  of  robotic  bar  code  technology  into  the 
pharmacy  at  MMC  had  significant  impact  on  staff,  processes,  and  the  overall  atmosphere  of  the 
department.  Upon  examination  of  the  data  collected  in  2011  (PRE)  compared  with  data  collected  in 
2013  (POST),  multiple  workflow  process  alterations  have  been  noted.  The  two  highest  time  consuming 
activities,  filling  and  dedication  order  entry,  were  seemingly  unaffected  by  the  technology,  as  well  as 
time  spent  loading  medications  into  the  Pyxis  machines. 

An  expected  benefit  of  automation  is  the  consolidation  of  inventory  in  known  locations  to  staff. 
Pharmacy  technicians  spent  nearly  the  same  amount  of  time  filling  medication  orders  between  the  PRE 
and  POST  time  periods.  Though  one  might  anticipate  a  decrease  in  time  spent  filling  medications  due  to 
automation  despite  a  fairly  consistent  patient  census  between  the  time  periods,  the  automation  of 
processes  did  not  significantly  impact  the  time  for  drug  dispensation  (i.e.  filling). 

The  results  showed  a  decrease  in  the  amount  of  time  spent  crediting  inventory;  likely  due  to  the 
automation  process  now  employed  to  complete  this  task  and/or  the  distribution  of  the  prescribed 
quantity  and  type  of  drug(s)  dispensed  to  the  clinical  units.  A  reduction  in  phone  calls  and  in  handling 
paper  (Fax,  Print,  Copy,  Notating)  had  been  shown  during  the  POST  period.  This  implies  fewer  calls  from 
clinical  units  (e.g.  nurse)  were  received  questioning  the  status  of  ordered  medications,  as  well  as  less 
hand-off  communications  (i.e.  paper  handling).  A  reduction  in  physical  walking  has  also  been  realized, 
implying  technicians  are  spending  less  time  traversing  to  fill  medication  orders.  Such  is  also 
demonstrative  of  the  overarching  role  of  the  robotic  technology -to  fill  patient  prescription  orders  via 
automation.  The  increase  in  personal  time  (e.g.  meal  break)  during  the  POST  assessment  was  seen  as  a 
positive  outcome  from  the  perspective  of  employee  satisfaction  and  perhaps  increased  task  completion 
efficiency,  as  pharmacists  and  technicians  were  seemingly  able  to  more  consistently  take  scheduled 
breaks  during  shifts. 

Examination  of  the  data  also  revealed  the  following: 

•  A  small  decrease  in  time  spent  for  order  entry.  This  was  expected  because  this  process  was  not 
primarily  impacted  by  technology  upgrades  or  automation.  Computerized  physician  order  entry 
(CPOE),  piloted  by  a  small  group  of  Conemaugh  physicians  (not  included  in  MIDHT  scope),  was 
anticipated  to  increase  efficiencies,  but  failed  to  be  observed  during  workflow  shadowing. 

•  A  small  increase  in  time  spent  checking  medications.  This  result  was  surprising  because 
procedurally  only  10%  of  medications  dispensed  from  the  robot  are  checked  by  a  pharmacist 
(POST)  whereas  100%  of  all  medications  manually  filled  by  technicians  were  reviewed  in  the  PRE 
period. 

Efficiency  gains  were  likely  not  quantitatively  realized  for  this  process  however.  Fewer  disruptions  would 
allow  staff  to  complete  tasks  at  hand.  During  the  PRE  observations  many  staff  did  not  break  for 
mealtime,  etc.  since  the  backlog  of  work  was  always  present.  While  backlog  in  the  POST  was  still 
present,  staff  had  more  confidence  in  the  processes  and  cooperation  between  each  other  to  actually 
take  a  break. 

In  addition  to  the  specific  discussions  in  regards  to  the  pharmacy  workflow  analyses  of  the  PRE  (2011) 
and  POST  (2013)  periods,  it  must  be  realized  that  in  addition  to  significant  daily  operational  changes,  the 
organizational  leadership  within  the  pharmacy  also  experienced  changes  during  this  time  period.  In  as 
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much,  differences  not  only  in  daily  operations,  but  also  workgroup  demeanor,  have  likely  influenced 
results. 

1.2.4  Robotic  Dispensing  to  MYMC 

Figure  11  displays  the  number  of  robotic  dispenses  per  month  that  were  sent  via  courier  to  rural  MYMC 
50  minutes  to  the  south.  Despite  having  no  in-house  pharmacy,  MYMC  was  fully  integrated  into  the 
BCMA  process  with  support  from  the  flagship  hospital  MMC. 


Robotic  Dispenses  to  Meyersdale 
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Figure  11.  Robotic  Dispensing  to  MYMC. 


1.2.5  Medication  Expenditures  and  Year-Ending  Inventory 

The  raw  data  consisting  of  medication  expenses  (US  $)  for  MMC  provided  by  the  organization's  Finance 
Department  [(FY  2009  -  2011  (PRE)  and  FY  2012  -  2013  (POST)]  was  converted  using  two  different 
indices:  average  annual  percent  change  of  brand  drug  prices  published  by  AARP,  Inc.  and  inflation  as 
reported  by  the  US  Bureau  of  Labor  Statistics  (PRNewswire-USNewswire  n.d.).  The  percent  change  price 
index  most  likely  better  reflects  the  true  increase  in  medication  expense  over  time,  as  the  prescription 
drug  expenditure  contribution  to  the  CPI-U  (inflation)  is  quite  small  -  typically  less  than  1%.  Therefore, 
the  results  for  the  analysis  of  medication  expenses  adjusted  using  the  percent  change  price  index 
follows.  An  independent-samples  t-test  was  used  to  determine  if  a  statistically  significant  difference 
existed  between  the  mean  medication  expense  for  the  pre  (n=12)  and  post  (n=8)  time  periods.  The  data 
did  not  exhibit  outliers  and  was  sufficiently  normal  for  each  level  of  time  period  as  assessed  by 
inspection  of  box  plots,  Q.-Q  plots,  and  the  Shapiro-Wilk's  test  (p  >  .05).  Flomogeneity  of  variances  held, 
as  assessed  by  Levene's  test  for  equality  of  variances  (p  =  .611).  As  further  evidence  of  homogeneity  of 
data  on  time  period,  a  t-test  revealed  that  MMC  medication  dosage  data  showed  no  statistically 
significant  mean  difference  between  time  periods  (post  -  pre),  Mdif=  5,281  +/-  23,350,  p  =  .6443.  The 
mean  difference,  Mdif,  of  medication  expense  (post  -  pre)  was  found  to  be  a  statistically  significant 
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decrease  at  alpha  =  .05,  Mdif=  -  $713,596.33,  SE  =  $87,459.31,  95%  Cl  [$529,851.13,  $897,341.52],  t(18) 
=  8.159,  p  <  .0005,  d  =  .19.  This  result  should  be  viewed  with  some  caution  as  the  AARP  source  is  retail- 
based  although  MMC  typically  dispenses  75-80%  brand  name  medications. 

An  independent  samples  t-test  was  attempted  on  the  change  in  the  ending  drug  inventory  value  (from 
MMC  Balance  Sheet  FY  2009  -  2013)  from  Pre  (n=3)  to  Post  (n=2).  Although  these  sample  sizes  are 
insufficient  for  valid  statistical  analysis,  the  mean  difference,  adjusting  using  percent  change  of  price,  is 
most  probably  not  statistically  meaningful,  Mdif  =  -  $52,327.90. 


time  point  by  QTR:  pre::  FY2009  through  FY2011; 
POST::  FY2012  through  FY2013 


pre  POST 

Time  Period 


Figure  12.  Linear  Regression  with  95%  Cl  curves  of  mean,  data 
adjusted  annual  percent  change  of  brand  name  drug  prices. 


Figure  13.  Box  Plot,  data  adjusted  using  average 
using  average  annual  percent  change 
of  brand  name  drug  prices. 


Figure  14.  Means  Plot,  data  adjusted  using  average  annual  percent  change  of  brand  name  drug  prices. 
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1.2.6  Non-returnable  Expired  Medications  -  MMC  &  MYMC 


The  quantity  and  cost  of  expired  medications  were  compared  by  time  period  for  MMC  &  MYMC.  Data 
was  retrieved  electronically  from  Guaranteed  Returns,  the  hospital's  vendor  that  handles  expired 
medications.  An  independent  t-test  revealed  no  statistically  significant  difference  between  time  periods 
for  the  mean  cost  or  quantity  of  expired  medications.  Please  refer  to 

Table  3.  The  distribution  of  both  cost  and  quantity  by  time  period  was  also  compared  using  the  Mann- 
Whitney  U  test;  the  distributions  were  found  not  to  be  statistically  significantly  different.  The  vendor 
promised  reduction  in  expired  medications  was  not  our  experience.  MIMC  was  removed  from  the  scope 
as  the  pharmacy  robotics  implementation  did  not  have  a  large  impact  on  operations  whereas  MMC 
pharmacy  directly  supports  MYMC. 


Time  period 

N 

Mean 

Std.  Deviation 

Std.  Error  Mean 

Quantity 

Pre 

19 

466.26 

205.542 

47.155 

Post 

12 

514.25 

91.844 

26.513 

Cost,  USD 

Pre 

19 

$18,726.00 

$12,294.76 

$2,820.61 

Post 

12 

$19,936.50 

$10,391.10 

$2,999.65 

Table  3.  Non-returnable  Expired  Medications. 


1.2.7  MMC  Pharmacy  Call  Volume  (Re-budget  Item) 

From  the  time  of  request  to  the  Telecommunications  Department,  16  months  of  data  was  available/held 
in  the  reporting  system.  Although  data  before  the  Pharmacy  system  vlO.l  upgrade  is  not  displayed. 
Figure  15  does  not  support  a  reduction  in  incoming  phone  calls  to  the  MMC  Pharmacy  during  the  Post 
period.  Phone  calls  to  the  Employee  Pharmacy  section  were  removed.  Outgoing  phone  calls  to  internal 
departments  (e.g.  Nursing)  are  not  collected  by  the  reporting  system. 
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Incoming  Phone  Calls  To  Pharmacy 
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Figure  15.  Incoming  Phone  Calls  to  MMC  Pharmacy. 


1.2.8  Nursing  Satisfaction  Survey 

The  Medication  Administration  System-Nurses'  Assessment  of  Satisfaction  (MAS-NAS),  developed  by 
Hurley  et  al  (Hurley  A  2003)  at  Brigham  and  Women's  Hospital,  was  made  available  through  various 
means  to  all  CHS  inpatient  nurses  in  early  April  2011  (Appendix  E).  The  overall  response  from  staff  had 
resulted  in  262  acceptable  surveys,  an  approximate  response  rate  of  38%.  Figure  16  depicted  nurse 
satisfaction  with  the  manual/paper  medication  administration  system  (baseline).  As  depicted,  46%  of 
respondents  were  satisfied  with  the  current  system.  This  presented  a  real  opportunity  to  improve 
nursing  satisfaction  through  the  pharmacy  robotics  and  BCMA  implementation.  The  survey  was  closed 
for  MMC  and  rural  MYMC  inpatient  nurses  but  remained  open  longer  for  nurses  at  MIMC.  Union 
negotiations  at  MIMC  were  a  barrier  to  receiving  additional  responses  from  that  facility.  Baseline  survey 
collection  at  MIMC  was  completed  in  January  2012. 
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Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 

Please  select. 
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1  -Completely  3  5  -  Me ither satisfied  7  9 

Dissatisfied  not  d  issatisf  ied 


Figure  16.  CHS  Nursing  Satisfaction  with  Medication  Administration  System  (Baseline). 
MMC  Site 


The  POST  Medication  Administration  System  Survey  -  Nurses'  Assessment  Survey  was  made  available  to 
all  inpatient  nursing  and  related  staff  at  Memorial  Medical  Center  in  early  March  2012  (3  months  POST). 
A  token  incentive  was  available  to  staff  for  properly  completed  surveys.  Communication  methods 
included  an  article  in  the  employee  newsletter,  direct  communications  with  nursing  managers  and 
placement  of  the  survey  link  on  the  Conemaugh  intranet  page.  The  objective  was  to  obtain  qualitative 
feedback  from  BCMA  users  using  a  validated  survey  instrument.  Survey  distribution  (POST)  at  MYMC 
was  completed  in  July  2012  (n=28).  A  second  POST  distribution  (~6  months  POST)  at  MMC  began  in 
September  2012  (n=277). 

Statistical  analyses  for  all  three  MMC  survey  data  sets  (PRE,  POST  1,  POST  2)  are  as  follow: 

Q30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 

Q26: 1  am  more  satisfied  with  this  new  medication  administration  system  than  with  the  previous  one. 

Taken  together,  Q30  and  Q26,  provide  an  overall  picture  of  the  satisfaction  of  medication  administration 
users.  An  ANOVA  over  the  three  time  periods  for  Q30  yields  a  significant  omnibus  (between  groups)  p- 
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value,  0.001.  The  mean  satisfaction  decreases  from  Baseline  to  Post  1  and  slightly  increases  for  Post  2. 
Pairwise  analysis  yields  significant  difference  between  Baseline  and  Post  1  (p-value,  0.001)  and  Post  2  (p- 
value,  0.011)  with  no  significance  between  post  time  periods.  The  mean  satisfaction  hovers  under  a 
value  of  5,  neither  satisfied  nor  dissatisfied.  The  change  in  satisfaction  is  depicted  visually  in  Figure  17. 
Q30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 


|  Q30  Descriptives  | 

Std. 

Std. 

95%  Confidence  Interval  for  Mean 

N 

Mean 

Deviation 

Error 

Lower  Bound 

Upper  Bound 

Minimum 

Maximum 

Baseline 

218 

5.66 

1.983 

.134 

5.40 

5.93 

1 

10 

Post  1 

285 

4.98 

2.310 

.137 

4.71 

5.24 

1 

10 

Post  2 

236 

5.08 

2.243 

.146 

4.80 

5.37 

1 

10 

Total 

739 

5.21 

2.213 

.081 

5.05 

5.37 

1 

10 

II  NOTE:  1  =  Completely  Dissatisfied  <> 

5  =  neither  <>  10  =  Completely  Satisfied 

|  Homogeneity  of  Variance  is  not  preserved  between  groups,  Levene  Statistic  p-value  =  .039 

Table  4.  Question  30  Descriptives 


Q30  Multiple  Comparisons 


Dependent  Variable:Q30 


Multiple 

Comparison 

Adjustment 

(1)  Time 
Period 

(J)  Time 
Period 

Mean 

Difference 

Std.  Error 

Sig. 

95%  Confidence  Interval 

(i-j) 

Lower  Bound 

Upper  Bound 

Baseline 

Post  1 

.685 

.192 

.001 

.23 

1.14 

Post  2 

.576 

.198 

.011 

.11 

1.04 

Games- 

Post  1 

Baseline 

-.685 

.192 

.001 

-1.14 

-.23 

Howell 

Post  2 

-.109 

.200 

.848 

-.58 

.36 

Post  2 

Baseline 

-.576 

.198 

.011 

-1.04 

-.11 

Post  1 

.109 

.200 

.848 

-.36 

.58 

*.  The  mean  difference  is  significant  at  the  0.05  level. 

Table  5.  Question  30  Multiple  Comparisons. 
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Baseline 


Post  1 

Time  Period 


Post  2 


Figure  17.  Question  25: 1  am  more  satisfied  with  this  new  medication  system,  BCMA,  than  with  the  previous  one. 


Q26  Descriptives 


N 

Mean 

Std. 

Deviation 

Std. 

Error 

95%  Confidence  Interval  for  Mean 

Minimum 

Maximum 

Notes 

Lower  Bound 

Upper  Bound 

Post  1 

288 

3.66 

1.613 

.095 

3.47 

3.85 

1 

6 

Post  2 

236 

3.63 

1.682 

.110 

3.41 

3.84 

1 

6 

Total 

524 

3.65 

1.643 

.072 

3.50 

3.79 

1 

6 

a.  Between-component  variance  is  negative.  It  was  replaced  by  0.0  in  computing  this  random  effects  measure. 

b.  Homogeneity  of  Variance  preserved  between  Post  1  and  2 


Table  6.  Question  26  Descriptives. 


Since  the  ANOVA  p-value,  0.821,  is  not  statistically  significant.  Post  1  and  Post  2  can  be  treated 
collectively  for  Q26.  The  mean  score  of  Q26  is  consistent  with  the  results  of  Q30  in  that  the  respondents 
do  not  favor  the  new  BCMA  system  over  the  original  system. 

Although  the  respondents  do  not  favor  BCMA  over  the  original  manual  system,  they  do  consider  BCMA 
to  be  safer  as  indicated  by  the  mean  value  of  Q24,  2.63,  between  moderately  and  slightly  agree. 

Q24:  This  is  a  safer  system  for  patients. 


Q24  Descriptives 


N 

Mean 

Std. 

Deviation 

Std. 

Error 

95%  Confidence  Interval  for  Mean 

Minimum 

Maximum 

Notes 

Lower  Bound 

Upper  Bound 

Post  1 

287 

2.67 

1.486 

.088 

2.50 

2.85 

1 

6 

Post  2 

237 

2.59 

1.512 

.098 

2.39 

2.78 

1 

6 

Total 

524 

2.63 

1.497 

.065 

2.51 

2.76 

1 

6 

a.  Between-component  variance  is  negative.  It  was  replaced  by  0.0  in  computing  this  random  effects  measure. 

b.  Homogeneity  of  Variance  preserved  between  Post  1  and  2 

Table  7.  Question  24  Descriptives. 


Since  the  ANOVA  p-value,  0.513,  is  not  statistically  significant.  Post  1  and  Post  2  can  be  treated 

collectively  for  Q24. 

Questions  4,  5,  6,  7, 13, 15,  and  18  can  be  interpreted  as  a  response  block  which  provides  insight  into 

the  perception  of  the  change-in  medication  reconciliation  and  safety. 

Q04:  Because  of  information  available  through  the  current  medication  administration  system  I  know 
both  the  intended  actions  and  side  effects  of  medications  I  administer. 

Q05: 1  find  the  drug  alert  feature  (drug/drug  or  drug/food  interaction)  of  the  current  medication 
administration  system  helpful. 

Q06:  The  current  medication  administration  system  makes  it  easy  to  check  active  medication  orders 
before  administering  medications. 

Q07:  The  current  medication  administration  system  provides  me  with  information  to  know  that  a 
medication  order  has  been  checked  by  a  pharmacist  before  I  administer  the  medication. 

Q13:  The  current  medication  administration  system  makes  it  easy  to  check  that  I  am  following  the  "5 
rights"  when  I  administer  medications. 
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Q.15:  The  current  medication  administration  system  is  effective  in  reducing  and  preventing  medication 
errors. 


Q18:  Information  available  through  the  current  medication  administration  system  helps  me  to  know 
what  to  do  should  my  patient  have  any  bad  reactions  from  a  medication. 


Omnibus  ANOVA  p-values  showed  significance 
for  Q04,  Q06,  Q07,  and  0.15.  Homogeneity  of 
variance  was  preserved  for  all  questions 
except  Q07. 

Multiple  pairwise  comparisons  by  time  period 
yield  significant  differences  between  Baseline 
and  at  least  one  Post  period.  Q04  shows 
improvement  over  baseline,  while  Q05,  Q06, 
and  Q15  display  a  decline. 


Question 

omnibus  ANOVA 
(betweeen  groups) 

MEAN 

p-value 

Baseline 

Post  1 

Post  2 

Q04 

0.001 

2.93 

3.18 

3.52 

Q05 

0.091 

2.87 

2.56 

2.73 

Q06 

0.036 

2.85 

2.77 

2.50 

Q07 

0 

2.91 

2.43 

2.40 

Q13 

0.169 

2.46 

2.34 

2.22 

Q15 

0.029 

3.35 

3.17 

2.97 

Q18 

0.408 

3.85 

3.68 

3.85 

Valid  N  (listw  ise)  »»»»»»»»> 

214 

275 

232 

Table  8.  Omnibus  ANOVA  for  Multiple  Questions. 


Multiple  Comparisons 

95%  Confidence  Interval 

Question 

(DV) 

Multiple 

Comparison 

Adjustment 

(1)  Time 
Period 

(J)  Time 
Period 

Mean 

Difference  (l-J) 

Std. 

Error 

Sig. 

Lower 

Bound 

Upper 

Bound 

Q04 

Sidak 

Baseline 

Post  1 

-0.252 

0.153 

0.268 

-0.62 

0.11 

Post  2 

-.595* 

0.16 

0.001 

-0.98 

-0.21 

Post  1 

Baseline 

0.252 

0.153 

0.268 

-0.11 

0.62 

Post  2 

-0.343 

0.152 

0.07 

-0.71 

0.02 

Post  2 

Baseline 

.595* 

0.16 

0.001 

0.21 

0.98 

Post  1 

0.343 

0.152 

0.07 

-0.02 

0.71 

Q06 

Sidak 

Baseline 

Post  1 

0.085 

0.136 

0.898 

-0.24 

0.41 

Post  2 

.349* 

0.142 

0.042 

0.01 

0.69 

Post  1 

Baseline 

-0.085 

0.136 

0.898 

-0.41 

0.24 

Post  2 

0.264 

0.135 

0.143 

-0.06 

0.59 

Post  2 

Baseline 

-.349* 

0.142 

0.042 

-0.69 

-0.01 

Post  1 

-0.264 

0.135 

0.143 

-0.59 

0.06 

Q07 

Games- Howell 

Baseline 

Post  1 

.476* 

0.135 

0.001 

0.16 

0.79 

Post  2 

.506* 

0.141 

0.001 

0.17 

0.84 

Post  1 

Baseline 

-.476* 

0.135 

0.001 

-0.79 

-0.16 

Post  2 

0.03 

0.122 

0.969 

-0.26 

0.32 

Post  2 

Baseline 

-.506* 

0.141 

0.001 

-0.84 

-0.17 

Post  1 

-0.03 

0.122 

0.969 

-0.32 

0.26 

Q15 

Sidak 

Baseline 

Post  1 

0.176 

0.134 

0.468 

-0.15 

0.5 

Post  2 

.374* 

0.14 

0.024 

0.04 

0.71 

Post  1 

Baseline 

-0.176 

0.134 

0.468 

-0.5 

0.15 

Post  2 

0.197 

0.132 

0.356 

-0.12 

0.51 

Post  2 

Baseline 

-.374* 

0.14 

0.024 

-0.71 

-0.04 

Post  1 

-0.197 

0.132 

0.356 

-0.51 

0.12 

*.  The  mean  difference  is  significant  at  the  0.05  level. 

Table  9.  Multiple  Comparisons  for  Questions  5,6,7  &  15. 
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Plot  of  Means 

Med  Rec  (Safety) 
Q04,  Q06,  Q07,  Q15 


Figure  18.  Medication  Safety  Questions  (Plot  of  Means). 


As  a  block,  the  questions  asking  solely  about  the  BCMA  system  show  a  rather  neutral  mean  response  as 
depicted  by  Figure  19.  This  is  consistent  with  the  results  discussed  earlier  for  overall  satisfaction. 


Plot  of  Means,  post 
BCMA  ONLY  questions 


Q23  024  025  026  027  028  029 

BCMA  Questions 


Figure  19.  BCMA  Questions  (Plot  of  Means). 
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Q23:  It  is  easier  to  do  all  the  checking  steps  needed  during  the  medication  administration  process. 

Q24:  This  is  a  safer  system  for  patients. 

Q25:  With  the  new  system,  it  is  easier  to  access  information  I  need  to  administer  medications. 

Q26: 1  am  more  satisfied  with  this  new  medication  administration  system  than  with  the  previous  one. 
Q27: 1  have  more  time  to  spend  with  patients. 

Q28:  Barcode/eMAR  has  made  the  medication  administration  process  more  efficient  for  me. 

Q29:  Medications  are  more  readily  available  when  I  need  them  for  patients. 

The  pairwise  (same  person  completing  survey  at  multiple  time  points)  analysis  is  as  follows: 

A  repeated-measures  analysis  of  variance  (RM-ANOVA)  was  completed  between  Baseline  and  Post  1  and 
Baseline  and  Post  2  periods  over  the  same  questions  as  were  analyzed  for  the  one-way  ANOVA.  The 
Sidak  formula  was  used  to  adjust  for  multiple  comparisons.  Overall  alpha  (family-wise)  remained  at 
0.05.  These  results  support  the  conclusions  of  non-pairwise  analysis.  Unfortunately  respondent  error 
with  creation  of  the  "SUM"  ID  and  a  lack  of  trust  by  nurses  in  order  to  protect  their  identity  prevented  a 
higher  sample  size. 


Time  Periods 

Question 

Later  time  - 

baseline 

Mean 

Difference 

over  TIME 

p-value 

Correction  for  non-sphericity 

Baseline  and  Post  1, 

N  =  35 

Q30 

decrease 

-1.143 

0.008 

Q04 

NS 

Q05 

NS 

Q06 

NS 

Q07 

decrease 

-0.788 

0.011 

Univariate  results, 
Greenhouse-Geisser 

Q13 

NS 

Q15 

NS 

Q18 

NS 

Baseline  and  Post  2, 

N  =  26 

Q30 

NS 

Q04 

increase 

0.846 

0.023 

Univariate  results, 
Greenhouse-Geisser 

Q05 

NS 

Q06 

NS 

Q07 

decrease 

-0.808 

0.034 

Univariate  results, 
Greenhouse-Geisser 

Q13 

NS 

Q15 

increase 

0.192 

0.019 

Univariate  results, 
Greenhouse-Geisser 

Q18 

NS 

NS  =  No  Significance 


Table  10.  Repeated  Measures  ANOVA. 
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Q30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 

Answer  Choices:  1  =  Completely  Dissatisfied  <>  5  =  neither  <>  10  =  Completely  Satisfied 

♦  means,  Baseline  =  5.63;  Post  1  =  4.49 

♦  The  results  above  indicate  a  statistically  significant  decrease  in  satisfaction  of  BCMA  from 
baseline  to  post  1. 

A  6-point  Likert  scale  was  used  for  the  remaining  questions  under  analysis: 

„  „  .  .  3  Slightly  Agree  5  Moderately  Disagree 

1  Strongly  Agree 

2  Moderately  Agree  4  Slightly  Disagree  6  Strongly  Disagree 
0  NA  Not  Applicable 

Please  refer  to  the  tables  below  for  quantification  of  the  mean  score  for  the  following  questions  by  time 
period. 

Q04:  Because  of  information  available  through  the  current  medication  administration  system  I  know 
both  the  intended  actions  and  side  effects  of  medications  I  administer. 

♦  The  results  above  indicate  a  statistically  significant  improvement  of  perception  of  knowledge 
after  implementation  of  BCMA  as  compared  to  baseline. 

Q07:  The  current  medication  administration  system  provides  me  with  information  to  know  that  a 
medication  order  has  been  checked  by  a  pharmacist  before  I  administer  the  medication. 

♦  The  results  above  indicate  a  statistically  significant  decrease  in  the  perception  of  the  knowledge 
that  the  medication  order  has  been  checked  by  a  pharmacist  before  administration  by  the  nurse 
after  implementation  of  BCMA  as  compared  to  baseline. 

Q15:  The  current  medication  administration  system  is  effective  in  reducing  and  preventing  medication 
errors. 

♦  The  results  above  indicate  a  statistically  significant  improvement  of  perception  of  the 
effectiveness  of  BCMA  to  reduce  and  prevent  medication  errors  as  compared  to  baseline. 
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Descriptive  Statistics:  Baseline(_l)  -  Post  1  (_2) 


Mean 

Std.  Deviation 

N 

Q07_l 

2.94 

1.694 

33 

Q07_2 

2.15 

.906 

33 

Descriptive  Statistics:  Baseline(_l)  -  Post  2  (_3) 


Mean 

Std.  Deviation 

N 

Q04_l 

2.73 

1.589 

26 

Q04_3 

3.58 

1.901 

26 

Q07_l 

3.31 

1.436 

26 

Q07_3 

2.50 

1.273 

26 

Q15_l 

3.65 

1.231 

26 

Q15_3 

3.00 

1.497 

26 

Table  11.  Descriptive  Statistics. 
MYMC  Site 


The  second  POST  survey  was  distributed  to  nursing  staff  at  MYMC  in  early  January  2013.  A  total  of  17 
surveys  were  completed  within  four  weeks,  with  two  of  those  surveys  not  100%  complete  so  the  sample 
size  varies  from  15-17  depending  on  question. 

Below  are  the  final  results  for  all  three  survey  distributions  at  MYMC: 

Responses  to  Questions  4-19  and  23-29  were  coded  0-6,  inclusive,  with  the  following  meaning: 


0  =  N/A  (not  applicable) 

1  =  Strongly  Agree 

4  =  Slightly  Disagree 

2  =  Moderately  Agree 

5  =  Moderately  Disagree 

3  =  Slightly  Agree 

6  =  Strongly  Disagree 

Q30  was  coded  from  1-10  with  a  between  response  interval  =  1,  where: 

1  =  completely  dissatisfied 5  =  neither  satisfied/dissatisfied,  10  =  completely  satisfied 

These  response  scales  are  ordinal  and  approximately  interval.  In  order  to  take  advantage  of  the  more 
powerful  parametric  tests,  the  scale  has  been  assumed  continuous.  As  this  is  an  assumption  for  analysis 
only,  non-parametric  tests  were  also  applied  to  check  consistency  of  the  analysis. 

The  following  analyses  are  between-group,  pre  vs.  post  1  vs.  post  2  study  time  periods,  with  a  family- 
wise  alpha  criteria  =  0.05.  When  homogeneity  of  variances  held,  the  Bonferroni  correction  was  applied 
for  pairwise  comparisons;  when  not,  Games-Howell  was  used. 
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Survey  Section  3:  Q04-08 


Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions. 

Only  Q04  showed  a  statistically  significant  omnibus  ANOVA,  p-value  =  0.024.  The  pairwise  results  follow 
and  show  significant  results  between  the  pre  and  post  1  periods,  p-value  =  0.037. 


Multiple  Comparisons 

Bonferroni 


Dependent  Variable  (1)  Study  Time  Period  (J)  Study  Time  Period 

Mean 

Differenc 

e  (l-J) 

Std. 

Error 

Sig. 

95%  Confidence 

Interval 

Lower 

Bound 

Upper 

Bound 

S3_Q04:  Because  of 

information  available 

through  the  current 

medication 

administration 

system  1  know  both 

the  intended  actions 

and  side  effects  of 

medications  1 

administer. 

PRE  (before  BCMA) 

N,PRE  =  19 

Post  1  (BCMA 
stabilization) 

-1.498* 

.579 

.037 

-2.93 

-.07 

Post  2  (BCMA) 

-1.480 

.641 

.073 

-3.06 

.10 

Post  1  (BCMA 
stabilization) 

N,Post  1  =  26 

PRE  (before  BCMA) 

1.498* 

.579 

.037 

.07 

2.93 

Post  2  (BCMA) 

.018 

.599 

1.000 

-1.46 

1.49 

Post  2  (BCMA) 

N, Post  2  =  17 

PRE  (before  BCMA) 

1.480 

.641 

.073 

-.10 

3.06 

Post  1  (BCMA 
stabilization) 

-.018 

.599 

1.000 

-1.49 

1.46 

*.  The  mean  difference  is  significant  at  the  0.05  level. 


Table  12.  Question  4  Statistical  Results. 
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Figure  20.  Graphical  Representation  of  Question  4  Results. 


Survey  Section  4:  Q09-19 

Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions  except  Q12  and  Q13. 
The  following  questions  showed  a  statistically  significant  omnibus  ANOVA: 


Question  p-value 

N,PRE 

N.Post  1 

N.Post  2 

Q10 

0.028 

19 

26 

17 

Q12 

<0.0005 

19 

26 

16 

Q13 

0.020 

18 

26 

16 

Q16 

0.025 

18 

25 

17 

The  pairwise  results  follow  and  show  significant  results  between: 
Question  p-value  pairwise  study  time  periods 


Q10 

0.033 

PRE  and  Post  1 

Q12 

<0.0005 

PRE  and  Post  1 

0.003 

PRE  and  Post  2 

Q13 

0.008 

PRE  and  Post  1 

0.048 

PRE  and  Post  2 

Q16 

0.017 

PRE  and  Post  1 
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Multiple  Comparisons 


Dependent  Variable 

(1)  Study  Time  Period 

(J)  Study  Time  Period 

S4_Q10:  The  drug 

information  available 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization 

Post  2  (BCMA) 

through  the  current 

medication 

Bonferroni 

Post  1  (BCMA 

PRE  (before  BCMA) 

administration  system 
is  easy  to  get  when  1 

stabilization) 

Post  2  (BCMA) 

need  that  information. 

Post  2  (BCMA) 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

S4_Q12:  The  current 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

Post  2  (BCMA) 

iTlcU  ILd  lIUll 

administration  system 
helps  me  to  be  efficient 

at  medication 

Games- 

Howell 

Post  1  (BCMA 
stabilization) 

PRE  (before  BCMA) 

Post  2  (BCMA) 

administration. 

Post  2  (BCMA) 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

S4  D1  T  The  rurrpnt 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

medication 

administration  system 
makes  it  easy  to  check 
that  1  am  following  the 
"5  rights"  when  1 

Games- 

Howell 

Post  1  (BCMA 
stabilization) 

Post  2  (BCMA) 

PRE  (before  BCMA) 

Post  2  (BCMA) 

administer 

medications. 

Post  2  (BCMA) 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

S4_Q16:  The  current 

Games- 

Howell 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

Post  1  (BCMA 

Post  2  (BCMA) 

medication 

administration  system 

PRE  (before  BCMA) 

is  user-friendly  to  the 

nurses  who  administer 

medications. 

stabilization) 

Post  2  (BCMA) 

Post  2  (BCMA) 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

Table  13.  Questions  10, 12, 13  &  16  Statistical  Results. 
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Figure  21.  Graphical  Representation  of  Question  10, 12, 13  &  16  Results. 

Survey  Section  7:  Q23-29 

Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions. 

No  questions  showed  a  statistically  significant  omnibus  ANOVA. 

Question  30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 

Homogeneity  of  variances  (over  study  time  periods)  held  for  this  question.  This  question  showed  a 
statistically  significant  omnibus  ANOVA,  p-value  =  0.023  and  indicates  a  trend  of  increasing 
dissatisfaction  from  pre  to  post  time  periods.  Figure  28  shows  the  dissatisfaction  at  MYMC  (rural 
facility)  was  slightly  worse  than  the  flagship  hospital,  MMC,  with  a  greater  disparity  in  satisfaction  at 
MYMC.  Please  note  the  sample  sizes  were  drastically  different. 


The  pairwise  results  follow: 
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Multiple  Comparisons 


Dependent  Variable  (1)  Study  Time  Period  (J)  Study  Time  Period 

Mean 

Difference 

(l-J) 

Std. 

Error 

Sig. 

95%  Confidence  Interval 

Lower 

Bound 

Upper 

Bound 

S_Q30:  Overall,  how 
satisfied  are  you  with 

the  current  medication 

administration  system? 

Bonferroni 

PRE  (before  BCMA) 

N,PRE  =  17 

Post  1  (BCMA 
stabilization) 

1.762 

.727 

.056 

-.03 

3.56 

Post  2  (BCMA) 

2.082* 

.819 

.042 

.06 

4.11 

Post  1  (BCMA 
stabilization) 

N,Post  1=25 

PRE  (before  BCMA) 

-1.762 

.727 

.056 

-3.56 

.03 

Post  2  (BCMA) 

.320 

.755 

1.000 

-1.55 

2.19 

Post  2  (BCMA) 

N,Post  2  =  15 

PRE  (before  BCMA) 

-2.082* 

.819 

.042 

-4.11 

-.06 

Post  1  (BCMA 
stabilization) 

-.320 

.755 

1.000 

-2.19 

1.55 

*.  The  mean  difference  is  significant  at  the  0.05  level. 

Table  14.  Question  30  Statistical  Results. 

Despite  the  overall  dissatisfaction  of  the  BCMA  system,  a  majority  of  nurses  (71%)  agreed  in  Figure  22 
positively  that  the  system  was  effective  in  reducing  and  preventing  medication  errors.  These  qualitative 
results  will  be  considered  in  regards  to  the  quantitative  analysis  of  error  counts/rates. 


The  current  medication  administration  system  is  effective  in  reducing  and  preventing 
medication  errors. 


Figure  22.  Nurse  Opinion  on  BCMA  Reducing  and  Preventing  Medication  Errors,  Post  2. 
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The  survey  asked  the  respondents  to  input  a  unique  code  ("SUM")  so  that  related-samples  analysis 
could  be  performed  over  the  three  time  periods,  baseline.  Post  1,  and  Post  2.  Unfortunately,  too  few 
respondents  followed  the  instructions  or  were  employed  during  all  time  points.  Only  six  responses  were 
common  between  baseline  and  Post  1,  seven  between  baseline  and  Post  2,  and  two  over  all  three  time 
periods.  This  sample  size  does  not  meet  the  generally  accepted  threshold  of  ten  for  analysis  to  be 
undertaken. 

In  conclusion,  nursing  staff  at  MYMC  is  less  satisfied  with  the  new  BCMA  system  than  the  previous 
manual  medication  administration  system.  The  second  post  survey  results  indicate  that  satisfaction 
continued  to  decline,  which  is  surprising  because  most  users  and  system  effectiveness  should  be  past 
the  learning  curve  phase  at  this  point.  On  the  other  hand,  these  results  are  not  surprising  because  users 
are  likely  a  close-knit  group  located  in  a  small,  rural  facility  and  may  be  resistant  to  technological 
advances.  As  demonstrated  by  the  comments  in  Table  7,  various  issues  are  present  and  contributing  to 
their  dissatisfaction. 

Comments  added  in  response  to  open-ended  survey  questions  during  the  Post  time  periods  have  been 
summarized  in  Table  15  below. 


Issue 

Count 

System  not  user  friendly 

11 

Medication  profile  errors 

6 

Barcodes  not  scanning 

4 

Dangerous  system 

4 

Communication  problems  between  pharmacy  and  nursing 

3 

Medications  not  easy  to  find 

2 

Easier  way  to  check  med  orders 

2 

Medication  info  not  contained  within  application 

1 

Takes  too  long  to  pass  medications 

1 

Timeliness  of  drugs  dispensed 

1 

Table  15.  Summary  of  Comments  Received. 

MIMC  Site 


The  same  validated  BCMA  survey  was  distributed  to  nursing  staff  at  MIMC  on  May  27,  2014  (3  months 
post-  live  deployment).  The  POST  survey  was  closed  on  June  13,  2014. 

MIMC  is  a  30-bed  community  hospital  serving  rural  northern  Cambria  County  and  surrounding  areas, 
located  approximately  45  minutes  from  Johnstown.  A  total  of  16  nurses  gave  responses  to  the  survey 
PRE  (2012)  and  30  staff  nurses  and  administrators  responded  to  the  survey  POST  BCMA  implementation 
(3  months).  All  respondents  to  the  POST  survey  questions  worked  on  the  unit  before  Admin-RX  was 
deployed.  Responses  below  are  reported  descriptively  because  questions  were  not  applicable  in  the  PRE 
survey.  The  majority  of  those  surveyed  responded  favorably: 

•  75%  agreed  it  is  easier  to  perform  all  the  checking  steps  needed  during  the  medication 
administration  process; 

•  more  than  90%  of  respondents  agreed  this  is  a  safer  system  for  patients; 

•  Approximately  80%  of  respondents  believed  it  is  easier  to  access  information  with  the  new 
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system,  which  implies  a  positive  impact  on  staff  efficiency  and  patient  safety. 

•  Approximately  80%  of  respondents  believed  it  takes  more  time  to  administer  medications  with 
the  new  system. 

•  Over  50%  of  staff  believed  BCMA  has  made  the  medication  administration  process  more 
efficient  for  them. 

The  last  results  are  somewhat  contradicted  by  the  previous  question  if  improved  efficiency  results  in 
spending  more  time  with  the  patient.  The  impact  of  pharmacy  automation  was  minimal  at  this  location 
as  an  in-house  pharmacy  is  present.  Again,  there  is  disagreement  amongst  staff  regarding  the  availability 
of  medications.  Most  respondents  probably  interpreted  this  question  to  be  about  ease  of  finding 
medications,  not  availability  of  product. 

As  Figure  23  displays,  satisfaction  is  spread  nearly  uniformly  across  all  answer  choices.  These  results  are 
similar  to  the  other  two  hospitals,  MMC  and  MYMC. 


Q27  I  have  more  time  to  spend  with 
patients. 

Answered:  23  Skipped:  7 


Strongly  Agree 


Moderately 

Agree 


Slightly  Agree 


Slightly 

Disagree 


Moderately 

Disagree 


Strongly 

Disagree 


Did  not  use 
old  system 


0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


Figure  23.  More  Time  to  Spend  With  Patients. 

The  same  survey  was  distributed  at  6  months  post  implementation  in  September  2014  to  the  nursing 
staff  (Post  2)  and  20  surveys  were  completed  by  the  nursing  staff.  Not  all  surveys  were  completed  in 
full,  therefore  the  sample  size  per  question  varied. 

Demographically,  no  statistically  significant  difference  was  found  for  the  mean,  median,  or  distribution 
of  the  following  variables  (see  Table  16).  Normality  was  violated  for  most  of  the  variables  over  most 
time  periods,  but  homogeneity  of  variance  held  for  all  but  'Year  at  CHS'.  An  ANOVA  was  used  to  test  the 
mean  (interpreted  with  some  caution  due  to  normality  violation),  the  Median  Test  for  the  median,  and 
the  Kruskal-Wallis  Test  was  applied  to  test  the  distribution. 
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Age, 

years 

Years  as  a 

Nurse 

Years  at 

CHS 

Hours 

worked/wk 

Pre 

N 

13 

15 

14 

15 

Mean 

50.08 

25.47 

13.29 

39.20 

Post  1 

N 

28 

28 

29 

28 

Mean 

46.11 

21.27 

8.69 

39.50 

Post  2 

N 

16 

19 

20 

20 

Mean 

46.13 

20.50 

7.40 

43.80 

Table  16.  Demographics  of  Survey  Respondents. 

Due  to  violations  of  both  normality  and  homogeneity  of  variance,  non-parametric  analyses  have  been 
applied  in  conjunction  with  a  non-pairwise  ANOVA  in  order  to  provide  results  with  the  most  depth 
possible.  Since  an  insufficient  number  of  surveys  were  able  to  be  matched  by  "SUM"  ID,  the  pairwise 
(same  person  completing  survey  at  different  time  points)  analysis  was  not  possible.  Unfortunately 
respondent  error  with  creation  of  the  "SUM"  ID  and  a  lack  of  trust  by  nurses  in  order  to  protect  their 
identity  prevented  a  sufficient  sample  size  to  execute  a  pairwise  analysis.  The  Sidak  formula  was  used  to 
adjust  for  multiple  tests.  Overall  alpha  (family-wise)  remained  at  0.05. 

Three  survey  questions  showed  a  statistically  significant  change  (pre  vs.  post)  in  Table  17. 

A  six-point  Likert  scale  was  used  for  these  questions. 


1  Strongly  Agree  3  Slightly  Agree  5  Moderately  Disagree 
2  Moderately  Agree  4  Slightly  Disagree  6  Strongly  Disagree 
0  NA  Not  Applicable 
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Kruskal-Wallis 

ANOVA, 

Test  on  the 

MEDIANS 

Median  Test, 

Survey  Question 

N 

Means 

p-values 

distribution  of 

p-values 

answers 

The  current  medication 

Pre  =  16 

Pre  =  4 

<  .0005 

.004 

Pre  =  4 

.025 

administration  system 
provides  me  with 

information  to  know 

Post  1  =  30 

Post  1  =  2 

(0,1):  .001 

(0,1):  .004 

Post  1=2 

(0,1):  .026 

that  a  medication 

order  has  been 

checked  by  a 
pharmacist  before  1 

administer  the 

medication. 

Post  2  =  20 

Post  2  =  2 

(0,2):  .002 

(0,2):  .017 

Post  2  =  2 

The  current  medication 

Pre  =  16 

Pre  =  4 

NS 

NS 

Pre  =4 

.016 

administration  system 
promotes  2-way 

communication 

Post  1  =  30 

Post  1  =  3 

Post  1=3 

(0,1):  .028 

between  clinicians 

(MD,  Pharmacist,  RN) 

about  medication 

orders. 

Post  2  =  20 

Post  2  =  3 

Post  2  =  3 

The  turnaround  time 

Pre  =15 

Pre  =5 

.002 

.003 

Pre  =5 

.001 

for  receiving 

medications  needed 

Post  1  =  30 

Post  1  =  3 

(0,1):  .001 

(0,1):  .002 

Post  1=2 

(0,1):  <.0005 

"stat"  or  for  patients 
newly  admitted  to  the 
unit  is  adequate. 

Post  2  =  20 

Post  2  =  3 

(0,2):  .032 

Post  2  =  3 

(0,2):  .017 

The  current  medication 

Pre  =  16 

Pre  =  4 

.001 

.008 

Pre  =3 

NS 

administration  system 
is  effective  in  reducing 
and  preventing 

Post  1  =  30 

Post  1  =  2 

(0,1):  .001 

(0,1):  .006 

Post  1=2 

medication  errors. 

Post  2  =  20 

Post  2  =  2 

(0,2):  .022 

Post  2  =  2 

Table  17.  p-values  for  Stated  Questions. 


NOTES: 

A.  (0,1)  and  (0,2)  refer  to  the  pairwise  comparison  between  Pre  and  Post  1  or  Post  2 

B.  Any  p-value  >  .013  and  <  .05  would  likely  be  statistically  significant  given  additional  sample  size; 
however,  when  the  more  stringent  critical  p-value  is  applied  adjusting  for  multiple  tests, 
statistical  significance  was  not  strictly  achieved. 

All  questions  show  approximately  a  33%  improvement  in  the  post  time  periods  when  compared  to  the 
pre,  shifting  from  the  'disagree'  to  the  'agree'  portion  of  the  answer  scale.  Graphical  comparison  of  the 
distribution  of  answers  by  time  period  follows  in  Figure  24  and  Figure  25: 


44 


The  current  medication  administration  system  provides  me 
with  information  to  know  that  a  medication  order  has  been 
checked  by  a  pharmacist  before  I  administer  the  medication. 


The  current  medication  administration  system  promotes  2-way 
communication  between  clinicians  (MD,  Pharmacist,  RN) 
about  medication  orders. 


Figure  24.  Pre,  Post  1  and  Post  2  Comparisons. 


The  turnaround  time  for  receiving  medications  needed  stat  or 
for  patients  newly  admitted  to  the  unit  is  adequate. 


■o 

CD 


«  -o 


The  current  medication  administration  system  is  effective  ii 
reducing  and  preventing  medication  errors. 


Figure  25.  Pre,  Post  1  and  Post  2  Comparisons. 

The  validated  BCMA  survey  by  Hurley  et  al  was  completed  by  nursing  staff  at  all  three  Conemaugh 
hospitals  (MMC,  MIMC  and  MYMC)  at  three  time  points  (Baseline,  Post  1  and  Post  2)  requisite  to  their 
implementation  dates.  Table  18  displays  the  sample  size  by  location  and  time  period  (varies  by 
question).  All  locations  show  a  trend  of  decreasing  satisfaction  over  time.  The  trends  of  MMC  and  MIMC 
are  more  similar  to  each  other  than  to  that  of  MYMC.  MMC  and  MIMC  began  slightly  satisfied  and 
finished  close  to  a  neutral  valuation;  whereas  MYMC  began  slightly  satisfied  and  finished  slightly 
dissatisfied.  This  finding  is  surprising  as  MMC  and  MYMC  are  tightly  coupled  from  an  operations 
standpoint  whereas  MIMC  has  an  in-house  pharmacy  and  relies  less  on  support  from  MMC. 
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Sample  Sizes 

time  period 

(max  possible) 

baseline 

Post  1 

POST  2 

Total 

Hospital 

MMC 

227 

288 

262 

777 

MYMC 

19 

26 

17 

62 

MIMC 

16 

30 

20 

66 

Total 

262 

344 

274 

905 

Table  18.  Max  Sample  Size  by  Location  and  Time  Period. 

Questions  23-29  with  the  following  Likert  response  scale  and  corresponding  numerical  quantification 
was  analyzed  using  a  two-way  ANOVA.  No  interaction  effect  was  found  between  the  two  factors, 
location  and  time  period.  The  time  period  main  effect  was  not  significant.  The  location  main  effect  was 
statistically  significant  for  all  but  Q25;  see  Table  19  for  details;  as  such,  the  differences  in  each  question 
between  levels  of  location  (MMC,  MYMC,  MIMC)  collapsed  across  time  period  (post  1,  post  2)  was 
statistically  significant  at  an  alpha  level  of  .05,  see  Table  19-  Contrast  Results. 

Q23.  It  is  easier  to  do  all  the  checking  steps  needed  during  the  medication  administration  process. 

Q24.  This  is  a  safer  system  for  patients. 

Q25.  With  the  new  system,  it  is  easier  to  access  information  I  need  to  administer  medications. 

Q26. 1  am  more  satisfied  with  this  new  medication  administration  system  than  with  the  previous  one. 
Q27. 1  have  more  time  to  spend  with  patients. 

Q28.  Barcode/eMAR  has  made  the  medication  administration  process  more  efficient  for  me. 

Q29.  Medications  are  more  readily  available  when  I  need  them  for  patients. 


Strongly 

Agree 

Moderately 

Agree 

Slightly 

Agree 

Slightly 

Disagree 

Moderately 

Disagree 

Strongly 

Disagree 

1 

2 

3 

4 

5 

6 

With  the  exception  of  Q26,  disagreement  increased  slightly  from  post  1  to  post  2.  Also,  by  time  period, 
MYMC  produced  a  score  greater  than  MMC  or  MIMC,  with  the  exception  of  Q25,  post  2.  Given,  with  the 
exception  of  MYMC,  that  the  scores  for  MMC  and  MIMC  were  in  the  agree  or  just  slightly  disagree 
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portion  of  the  scale  above  implies  that  the  BCMA  had  an  overall  slightly  positive  influence  on  medication 
administration. 


LOCATION 

TIME  PERIOD 

Question 

F 

df 

p-value 

F  df  p-value 

Tests  of 

23 

6.573 

2 

0.002 

NSS 

Between- 

24 

9.376 

2 

<.0005 

NSS 

Subjects 

25 

overall  by  factor, 
location 

NSS 

NSS 

Effects 

26 

or 

6.265 

2 

0.002 

NSS 

27 

time  period 

5.184 

2 

0.006 

NSS 

28 

5.155 

2 

0.006 

NSS 

29 

3.498 

2 

0.031 

NSS 

Pairs 

Contrast 

23 

MYMC  vs  MMC 

<  .0005 

Results 

24 

MYMC  vs  MMC 

<  .0005 

MYMC  vs  Miners 

<  .0005 

25 

NSS 

26 

MYMC  vs  MMC 

0.001 

MYMC  vs  Miners 

0.003 

27 

MYMC  vs  MMC 

0.004 

28 

MYMC  vs  MMC 

0.003 

29 

Miners  vs  MMC 

0.016 

MYMC  vs  Miners 

0.016 

NSS:  not  statistically  significant 

where:  MMC  =  Memorial  and  MYMC  =  Meyersdale 


Table  19.  Statistical  Results  for  Stated  Questions. 
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Estimated  Marginal  Means  Estimated  Marginal  Means  Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marqln.il  Means  of  Q?3 


Estimated  Marginal  Means  of  Q23 


Estimated  Marginal  Means  of  Q?4 


2.0- 


Memorial  Meyersdale  Miners 

Hospital 


Estimated  Marginal  Means  of  Q24 


Hospital 


Estimated  Marginal  Means  of  Q?fi 


Estimated  Marginal  Means  of  Q2G 


time  period 


Estimated  Marginal  Means  of  Q25 


Hospital 


Estimated  Marginal  Means  of  Q26 


Hospital 


Figure  26.  Estimated  Marginal  Means  for  Stated  Questions. 


Estimated  Marginal  Means  Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  Q27 


time  period 


Estimated  Marginal  Means  of  Q27 


Hospital 


Estimated  Marginal  Means  of  Q28 


time  period 


Estimated  Marginal  Means  of  Q28 


Hospital 


Estimated  Marginal  Means  of  Q29 


time  period 


Estimated  Marginal  Means  of  Q29 


Hospital 


Figure  27.  Estimated  Marginal  Means  of  Stated  Questions  II. 
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Question  thirty,  Q30,  with  the  following  Likert  response  scale  and  corresponding  numerical 
quantification  was  analyzed  using  a  two-way  ANOVA.  No  interaction  effect  was  found  between  the  two 
factors,  location  and  time  period.  The  Location  main  effect  was  not  significant.  The  time  period  main 
effect  was  statistically  significant,  F(2,849)  =  4.233,  p  =  .015. 

Q30.  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 


Completely 

Dissatisfied 

Neither 

satisfied 

nor 

dissatisfied 

Completely 

Satisfied 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

The  differences  in  overall  satisfaction  between  levels  of  time  period  (baseline,  post  1,  post  2)  collapsed 
across  location  (MMC,  MYMC,  MIMC)  was  statistically  significant  at  an  alpha  level  of  .05.  The  contrasts 
between  baseline  and  post  1  along  with  those  between  baseline  and  post  2  were  significant,  p  =  .016 
and  .007  respectively.  Figure  28  on  the  left  reveals  that  the  MYMC  location  likely  contributes  most  to 
this  finding.  All  locations  show  a  trend  of  decreasing  satisfaction.  The  trends  of  MMC  and  MIMC  are 
more  similar  to  each  other  than  to  that  of  MYMC.  MMC  and  MIMC  began  slightly  satisfied  and  finished 
close  to  a  neutral  valuation;  whereas  MYMC  began  slightly  satisfied  and  finished  slightly  dissatisfied. 

Estimated  Marginal  Means  of  Q30  Estimated  Marginal  Means  of  Q30 


time  period  Hospital 


Figure  28.  Estimated  Marginal  Means  of  Questions  30  by  Time  Period  and  Hospital. 

Conclusion 

A  strong  sample  size  (n  >  900)  was  received  from  Nursing  staff  at  MMC,  MYMC  and  MIMC  during  one 
PRE-BCMA  distribution  and  two  POST-BCMA  distributions  (3  and  6  months  post)  of  the  Medication 
Administration  System  Survey  -  Nurses'  Assessment  Survey  (MAS-NAS).  The  primary  question  related  to 
satisfaction  of  the  medication  administration  system  did  have  a  slight  negative  change  from  PRE  to  POST 
but  the  overall  opinion  across  time  periods  is  close  to  5  (neutral  feeling)  on  the  1-10  scale.  MYMC,  with  a 
much  smaller  sample  size,  had  a  greater  negative  trend  than  MMC  whereas  MIMC  was  more  similar  to 
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MMC.  Despite  the  overall  drop  in  satisfaction  of  the  BCMA  system,  a  majority  of  nurses  (>70%)  agreed 
positively  with  the  perception  that  the  system  was  effective  in  reducing  and  preventing  medication 
errors.  Demographics  of  the  sample  size  should  be  considered.  With  an  average  of  10.5  years  of  nursing 
experience  (56  nurses  with  over  20  years  of  experience)  and  >80%  of  respondents  never  have  used  a 
BCMA  system  before,  it  can  be  safely  assumed  that  staff  nurses  as  a  whole  did  not  positively  support  the 
change  from  a  satisfaction  perspective  during  the  evaluation  periods. 

1.2.9  School  of  Nursing  Satisfaction  Survey  (Re-budget  Item) 

An  abbreviated  MAS-NAS  baseline  survey  was  distributed  to  all  nursing  students  on  December  3,  2012 
(Appendix  F).  A  strong  response  was  received  (n=102)  before  the  survey  was  closed  on  December  14, 

2012.  Survey  respondents  that  followed  instructions  received  a  token  incentive  for  completing  a  valid 
survey.  Five  respondents  stated  in  written  comments  that  they  would  like  improved  access  to 
medication  carts  for  training  purposes. 

A  total  of  87  Conemaugh  nursing  students  completed  the  POST  BCMA  survey  during  April  and  May  of 

2013,  which  included  a  good  balance  between  1st  and  2nd  year  students.  The  following  select  questions 
suggest  a  positive  change  in  access  to  the  system  after  additional  medication  carts  were  made  available 
and  a  favorable  opinion  of  the  BCMA  system  on  improving  patient  safety. 

Q5  Current  status: 

Answered:  87  Skipped:  8 

1  st  Year 
Student 


2nd  Year- 
Student 


Figure  29.  Current  Status  of  Nursing  Student. 


Q7  Do  you  have  access  to  medication  carts 
(blue  Rubbermaid  with  drawers)  when  you 
need  them  for  training  on  the  floor? 

Answered:  87  Skipped:  0 


100% 

80% 


60% 

40% 

20% 

0% 


Frequently  Very  Often  Sometimes 


Not  Very 
Often 


Never 


Figure  30.  Access  to  Medication  Carts  (POST). 
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Q10  The  current  medication  administration 
system  is  effective  in  reducing  and 
preventing  medication  errors. 

Answered:  87  Skipped:  0 


100% 


Strongly  Moderately  Slightly  Slightly  Moderately  Strongly 

Agree  Agree  Agree  Disagree  Disagree  Disagree 


Figure  31.  Impact  on  Preventing  Medication  Errors. 

Nursing  students  did  not  suggest  any  real  change  in  patient  safety  deduced  from  the  same  median  and 
mode  for  both  time  periods  (Strongly  Agree).  Overall,  they  believe  the  BCMA  system  is  effective  is 
preventing  medication  errors  but  additional  system  usage  did  not  change  their  opinion. 

Q11  Overall,  how  satisfied  are  you  with  the 
current  medication  administration  system? 

Answered:  87  Skipped:  0 

100% 

80% 

60% 

40% 

20% 

0% 

0  -  1 
Complet 
ely... 


Figure  32.  Overall  Satisfaction  of  BCMA  System. 

Overall  satisfaction  did  not  change  over  time  with  a  median  of  9.0  for  both  PRE  and  POST  data  sets. 
Nursing  students  have  a  favorable  opinion  of  the  BCMA  system  and  likely  cannot  compare  historically  to 
the  manual  medication  administration  system  previously  used  at  Conemaugh. 

Conclusion 

A  strong  sample  size  (n  >180)  was  received  from  Conemaugh  nursing  students  before  and  after  the 
purchase  and  deployment  of  additional  blue  Rubbermaid  medication  carts  to  the  School  of  Nursing  to 
the  distribution  of  an  abbreviated  MAS-NAS.  The  sample  had  a  nice  balance  between  first  and  second 
year  students.  Comparison  of  mean  statistics  between  PRE  (Sometimes  -  2.75)  and  POST  (Very  Often  - 
3.21)  results  suggest  a  positive  change  occurred  and  ultimately  a  favorable  training  experience  was 
created  through  improved  access  to  the  BCMA  system. 
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1.2.10  Nursing  Workflow/Time  &  Motion  Study  -  All  Sites 


Study  team  personnel  physically  shadowed  the  clinician  (RN  or  LPN)  during  multiple  contiguous  four- 
hour  blocks  of  time  during  their  work  shift  before  (PRE)  and  after  (POST)  BCMA/eMAR  implementation 
at  all  sites  (MMC,  MYMC,  and  MIMC).  Prior  to  shadowing,  the  study  team  met  with  the  clinicians  to 
confirm  the  completeness  and  accuracy  of  the  activities  listed  on  the  time  and  motion  tool,  the  activity 
identification  (AID)  index,  as  sufficient  representations  of  the  corresponding  actual  workday  activities. 
Effort  was  made  to  obtain  a  representative  data  sample  by  varying  the  clinician,  the  clinical  unit  (e.g. 
patient  type  and  load),  and  time  of  day  of  the  observation.  As  used,  the  time  and  motion  tool  consisted 
of  35  unique  activities  (AIDs).  The  implementation  of  BCMA/eMAR  was  expected  to  alter  nursing 
procedures  particularly  at  the  task  (activity)  level.  Some  tasks  were  unaffected,  some  were  expanded  in 
scope,  while  others  were  reduced,  and  a  few  were  eliminated.  These  procedural  adjustments 
introduced  variation  from  PRE  to  POST  into  some  AIDs.  The  confounding  influence  of  this  variation  was 
managed  first  through  the  defining  of  the  AIDs  so  as  to  minimize  the  replacements  from  PRE  to  POST 
shadowing,  and  second  by  the  categorization  of  the  AIDs  into  work  processes.  The  presumption  being 
that  the  magnitude  of  any  change  at  the  AID  level  would  not  be  so  large  as  to  change  the  process 
category  of  that  activity.  For  analysis,  each  AID  was  placed  into  one  often  processes  (PIDs);  these 
processes  represent  collections  of  activities  of  related  function,  see  Table  21.  A  PID  is  a  stratification  of 
the  data  by  aggregating  every  instance  of  specific  AIDs  of  related-function.  Most  PIDs  contain  multiple 
AIDs. 

Every  time  study  personnel  observed  and  recorded  an  activity,  it  represented  an  instance  of  that  AID. 
When  the  observed  task  was  not  represented  by  a  predefined  AID,  that  instance  was  recorded  using  the 
'other'  AID.  The  time  and  motion  tool  recorded  the  elapsed  time  per  each  instance  of  an  AID.  The  sum 
of  the  duration  (time)  of  all  the  instances  of  AIDs  yielded  a  total  observation  time  per  time  period. 
Similarly,  the  sum  of  the  time  per  instance  of  the  AIDs  assigned  to  a  PID  yielded  the  total  time  per  PID. 
The  total  time  of  shadowing  PRE  and  POST  for  each  location  was  nearly  equal  (Table  20).  A 
normalization  procedure  was  therefore  applied  to  allow  for  legitimate  direct  comparisons  of  the  data 
between  time  periods  (by  location).  The  total  time  for  each  location  by  time  period  (PRE  and  POST)  was 
calculated  and  used  to  normalize  (rescale)  the  data.  The  elapsed  time  for  each  instance  of  AID  was 
divided  by  the  total  time  for  each  location.  The  normalized  data  was  then  multiplied  by  100  to  yield  the 
outcome  variable  of  analysis,  dTime_normd_pct. 


Total  Time 

Per  Location  and  Time  Period 


MMC 

MYMC 

MIMC 

Time 

Period 

hh:mm:ss 

seconds 

observations 

hh:mm:ss 

seconds 

observations 

hh:mm:ss 

seconds 

observations 

PRE 

105:22:10 

379,330 

26 

3:55:00 

14,100 

1 

4:04:46 

14,868 

1 

POST 

104:53:31 

377,611 

26 

4:15:58 

15,358 

1 

3:59:46 

14,386 

1 

difference 

between 

time 

periods 

0:28:39 

0:20:58 

0:08:02 

Table  20.  Nurse  Shadowing  Total  Time  by  Location. 

This  derived  variable  represents  the  percent  of  total  time  (per  location  per  time  period)  for  each 
instance  of  AID. 
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The  between  time  period  analysis,  PRE  vs.  POST,  for  each  location  was  applied  on  the  following  levels  of 
stratification: 

1.  process  (PID)  and 

a.  BCMA/eMAR  (and  related)  activities 

2.  individual  activity  (AID) 

3.  intra-location  geography  by  PID 

The  BCMA/eMAR  level  of  analysis  focuses  on  a  subset  of  AIDs  of  PID  1;  specifically  AIDs  4,  5, 14,  and  24. 
This  level  of  analysis  was  necessary  since  AIDs  4,  5,  and  14  were  directly  impacted  (replaced)  by  AID  24 
(BCMA/eMAR,  POST  only).  In  fact,  the  data  showed  that  AIDs  4,  5,  and  14  were  not  used  during  the 
POST  shadowing  and  therefore  effectively  eliminated.  The  tasks  represented  were  now  being 
accomplished  by  the  BCMA/eMAR  systems.  Therefore,  this  analysis  compared  the  aggregation  of  4,  5, 
and  14  in  the  PRE  with  all  instances  of  AID  24  (POST). 


Activity 

ID 

(AID) 

Process 

ID 

(PID) 

Process 

Description 

Individual  Activity  Description 

Process 

ID 

1 

Passing  medications 

1 

2 

Hang  IV fluids 

1 

3 

Retrieve  meds  from  Pyxis,  bins,  or  tube 

1 

4 

Compare  physical  med  to  paper  MAR 

1 

5 

1 

MED 

Confirm  "Five  Rights" 

1 

7 

administration 

Passing  IM/Subq  meds 

1 

8 

Passing transdermal  meds 

1 

9 

Stock  or  Retrieve  from  MED  cart 

1 

14 

Transcibe  med  orders  -  paper  MAR 

1 

24 

eMAR 

1 

11 

SCANning 

2 

19 

computer 

charting 

review  lab/rad  results  -  computer 

2 

21 

2 

Admissions 

2 

22 

Discharges 

2 

23 

Standard 

2 

51 

patient-related 

activities 

Pt  assessment-e.g.  vitals 

3 

52 

3 

Physician  Rounding-listening 

3 

53 

Pt  assistance-e.g.  meals 

3 

64 

talking  in-person 
to::> 

Colleague/Staff  for  Pt. 

4 

66 

4 

verbal  report-out  with  nurses 

4 

71 

Patient/Family 

4 

76 

5 

MED-related  interruptions  ONLY 

5 

73 

6 

waiting 

waiting  for  MED  cart 

6 

74 

waiting  for  Pxyis  access 

6 

72 

7 

walking  inside 

7 

79 

Patient  (patient  family) 

8 

80 

Doctor 

8 

81 

8 

Phone 

Nurse 

8 

82 

Pharmacy 

8 

85 

misc. 

8 

20 

fax,  copy 

11 

55 

9 

Paper 

check  or  write  PAPER  charts 

11 

56 

review  lab/rad  results  -  PAPER 

11 

70 

Nurse's  note  for  charting  later 

11 

62 

10 

personal  time-Any 

10 

Table  21.  Table  14.  Nurse  Shadowing:  Process  (PID)  Details  . 
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The  AID  analysis  level  is  the  aggregation  of  every  instance  of  each  AID  used  to  create  the  PIDs.  Intra¬ 
location  geography  is  a  stratification  by  unit  within  MMC,  namely: 

•  A9  =  Ashman/Rose  Pavilion,  clinical  unit  9,  Ashman  section 

•  RIO  =  Ashman/Rose  Pavilion,  clinical  unit  10,  Rose  section 

•  GS5  =  Good  Samaritan  Building,  clinical  unit  5 

•  MS7  =  Medical  Surgical  clinical  unit  7 

These  clinical  areas  were  chosen  as  a  representative  sample  of  MMC  based  upon  physical  layout  and 
patient  composition.  Given  the  size  of  MYMC  and  MIMC  (20-bed  rural  critical  access  hospital  and  thirty 
bed  facility  respectively),  the  intra-location  geography  concept  is  not  applicable. 

ANOVA  analyses,  between  time  period  (PRE  vs.  POST),  was  accomplished  for  each  location  separately. 
Applying  an  ANOVA  analysis  to  dTime_normd_pct  provides  the  most  effective  manner  in  which  to 
simultaneously  account  for  both  intrinsic  characteristics  of  this  dataset,  sum  of  time  and  frequency,  that 
influence  the  quantification  of  efficiency  (productivity)  and  gross  system  changes.  Analysis  of  the  mean 
of  dTime_normd_pct  accounts  for  both  components  and  provides  directly  comparable  results  across 
time  periods.  A  family-wise  alpha  =  0.05  was  set  as  the  criteria  for  statistical  significance. 


MMC 


The  time  and  motion  POST  data  collection  occurred  (January- April  2012)  at  MMC.  This  data  was 
compared  to  the  observation  in  the  PRE  period  (May  -  June  2011)  with  the  following  results. 

The  statistically  significant  finding  on  the  BCMA/eMAR  level  implies  that  the  implementation  is  being 
utilized  as  intended.  Also,  since  no  PID  individually  and  only  one  AID  individually,  which  likely  would  not 
be  influenced  by  BCMA  implementation,  showed  statistical  significance,  the  results  imply  that  the 
implementation  of  bar  coded  medication  administration  did  not  positively  or  negatively  impact  the 
efficiency  of  the  nursing  workflow  at  MMC  as  analyzed.  Please  refer  to  Figure  33  and  Table  22  and  Table 
23  for  details. 
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NurseShadowing:  Summary  of  Percentages 
Location  =  MMC 


13  PRE  ^i  POST  =®=  cumulative %,  PRE  —A— cumulative  %,  POST 


(PID):  Description 


Note:  Cumulative  percent  does  not  reach  100%  because  the  "Other"  PID  is  not  graphed  since  these  activities  are  not  germane 
to  the  analyses. 

Figure  33.  Nurse  Shadowing:  Summary  of  Percentages,  MMC. 


MMC  PRE  (N  =  26  observations  -  RN): 


Nurse  Shadowing:  Study  Time  Period  ■ 

=  PRE, 

Location  =  MMC 

Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh.mm.ss 

cumulative 

%  of  time 

%of 

Total  Time 
per  PID 

Total  Time 
/PID, 

hh:mm  :ss 

PID 

PID 

Count  of 
Activity 

IDs 

avg.  time  / 
AID, 

hh:mm:ss 

4 

20:48:13 

19.7% 

19.7% 

20:48:13 

4 

4 

1290 

0:00:58 

1 

37:50:29 

35.9% 

16.2% 

17:02:16 

1 

7 

1217 

0:00:33 

2 

53:28:41 

50.8% 

14.8% 

15:38:12 

2 

1 

987 

0:01:02 

3 

68:09:45 

64.7% 

13.9% 

14:41:04 

3 

9 

600 

0:00:50 

7 

79:14:10 

75.2% 

10.5% 

11:04:25 

7 

3 

559 

0:01:35 

9 

87:35:16 

83.1% 

7.9% 

8:21:06 

9 

2 

252 

0:03:43 

10 

92:57:44 

88.2% 

5.1% 

5:22:28 

10 

8 

227 

0:01:12 

8 

97:30:48 

92.5% 

4.3% 

4:33:04 

8 

10 

90 

0:03:35 

6 

98:30:09 

93.5% 

0.9% 

0:59:21 

6 

6 

51 

0:01:10 

5 

98:37:32 

93.6% 

0.1% 

0:07:23 

5 

5 

13 

0:00:34 

Table  22.  CONEMAUGH  PRE:  Details  by  Process. 
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MMC  POST  (N  =  26  observations  -  RN): 


Nurse  Shadowing:  Study  Time  Period  = 

■POST, 

Location  =  MMC 

Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative  ,  . 

..  cumulative 

time,  „  . 

..  %  of  time 

hh:mm:ss 

%  of 

Total  Time 
per  PID 

Total  Time 
/PID, 

hh:m  m  :ss 

PID 

PID 

Count  of 
Activity 

IDs 

avg.  time  / 
AID, 

hh:mm:ss 

1 

22:10:21 

21.1% 

21.1% 

22:10:21 

1 

7 

1261 

0:00:33 

2 

41:42:02 

39.8% 

18.6% 

19:31:41 

2 

1 

1249 

0:01:04 

4 

60:07:50 

57.3% 

17.6% 

18:25:48 

4 

4 

1109 

0:01:00 

3 

72:12:12 

68.8% 

11.5% 

12:04:22 

3 

9 

598 

0:00:55 

7 

83:46:53 

79.9% 

11.0% 

11:34:41 

7 

3 

491 

0:01:29 

9 

92:56:34 

88.6% 

8.7% 

9:09:41 

9 

2 

363 

0:03:14 

8 

98:28:14 

93.9% 

5.3% 

5:31:40 

8 

8 

221 

0:01:30 

10 

100:57:21 

96.2% 

2.4% 

2:29:07 

10 

10 

39 

0:03:49 

6 

101:05:17 

96.4% 

0.1% 

0:07:56 

6 

6 

8 

0:00:59 

5 

101:07:23 

96.4% 

0.0% 

0:02:06 

5 

5 

3 

0:00:42 

Table  23.  MMC  POST:  Details  by  Process. 


PRE  vs  POST 

PID 

ri/PRE 

ri/posT 

p-value 

Change  of  the  meanPOsT  relative 

to  that  of  the  PRE 

PIDs,  individually 

No  statistical  significance  was  found. 

BCMA/eMAR 

1 

(subset  of) 

432 

681 

<  0.0005 

Increase 

Intra¬ 

location 

geography 

by 

PID 

A9 

1 

2 

475 

124 

512 

182 

0.013 

0.005 

Increase 

Decrease 

GS5 

No  statistical  significance  was  found. 

MS7 

4 

7 

269 

229 

264 

284 

0.045 

0.027 

Decrease 

Decrease 

R10 

1 

270 

342 

0.009 

Increase 

AIDs,  individually 

AID  3 

1 

238 

179 

<  0.0005 

Decrease 

Table  24.  Summary  of  Statistical  Findings  -  MMC. 

Also,  for  MMC,  analyses  were  conducted  on  average  daily  census  (ADC),  case  mix  index  (CMI)  and 
between  intra-location  geography.  PRE  was  compared  with  POST.  Given  the  sample  size  for  the  PRE 
time  period,  Mann-Whitney  test  was  used.  No  statistical  significance  was  found  overall  for  the  analysis 
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by  time  period.  The  data  set  was  stratified  by  intra-location  geography  and  repeated.  No  statistical 
significance  was  found.  The  between  intra-location  geography  ANOVA  produced  a  significant  contrast 
only  between  A9  and  GS5  (p-value  =  0.029)  indicating  that  these  two  clinical  units  differ  from  each  other 
on  dTime_normd_pct.  No  evidence  was  found  to  imply  that  this  result  impacted  the  conclusions  drawn 
overall. 

MYMC 


The  time  and  motion  POST  data  collection  occurred  (August  2012)  at  MYMC.  This  data  was  compared  to 
the  observation  in  the  PRE  period  (June  2011)  with  the  following  results. 

Statistical  significance  was  found  for  two  groupings,  'MED  administration'  and  'Talking  in-person',  but  no 
individual  activities.  Statistical  significance  was  found  for  BCMA/eMAR  level  as  well,  implying  that  the 
implementation  is  being  successfully  utilized  as  intended.  The  findings  must  be  viewed  with  caution 
since  the  data  contains  observations  from  only  one  clinician  for  one  four-hour  time  period,  which  is 
insufficient  to  be  a  representative  sample;  an  unavoidable  consequence  of  the  typically  low  patient 
census  at  this  location.  If  the  significance  would  hold  for  a  more  representative  sample,  the  implication 
could  be  that  BCMA  has  some  influence  at  MYMC.  Please  refer  to  Figure  34  and  Table  25  and  Table  26 
and  for  details. 


Nurse  Shadowing:  Summary  of  Percentages 
Location  =  MYMC 


PRE 


I  POST 


cumulative  %,  PRE 


-cumulative  %,  POST 


(PID):  Description 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 


Note:  Cumulative  percent  does  not  reach  100%  because  the  "Other"  PID  is  not  graphed  since  these  activities  are  not  germane 
to  the  analyses. 

Figure  34.  Nurse  Shadowing:  Summary  of  Percentages,  MYMC. 
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MYMC  PRE  (N  =  1  observation  -  LPN): 


Nurse  Shadowing:  Study  Time  Period  =  PRE,  Location  =  MYMC 


Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of 

Total  Time 
per  PID 

Total  Time 
/  PID, 

hh:mm  :ss 

PID 

PID 

Count  of 
Activity 
IDs 

avg.  time  / 
AID, 

hh:mm:ss 

10 

1:06:28 

28.3% 

28.3% 

1:06:28 

10 

1 

35 

0:00:55 

3 

1:53:05 

48.1% 

19.8% 

0:46:37 

3 

7 

17 

0:00:41 

4 

2:27:58 

63.0% 

14.8% 

0:34:53 

4 

4 

16 

0:02:11 

1 

3:00:04 

76.6% 

13.7% 

0:32:06 

1 

3 

7 

0:06:40 

7 

3:11:43 

81.6% 

5.0% 

0:11:39 

7 

10 

6 

0:11:05 

9 

3:20:48 

85.4% 

3.9% 

0:09:05 

9 

9 

5 

0:01:49 

2 

3:23:09 

86.4% 

1.0% 

0:02:21 

2 

8 

2 

0:00:54 

8 

3:24:57 

87.2% 

0.8% 

0:01:48 

8 

2 

1 

0:02:21 

5 

3:25:23 

87.4% 

0.2% 

0:00:26 

5 

5 

1 

0:00:26 

6 

3:25:23 

87.4% 

0.0% 

0:00:00 

6 

6 

0 

Table  25.  MYMC  PRE:  Details  by  Process. 
MYMC  POST  (N  =  1  observation  -  LPN): 


Nurse  Shadowing:  Study  Time  Period  =  POST,  Location  =  MYMC 


Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of 

Total  Time 
per  PID 

Total  Time 
/  PID, 

hh:mm  :ss 

PID 

PID 

Count  of 
Activity 
IDs 

avg.  time  / 
AID, 

hh:mm:ss 

3 

1:42:04 

39.9% 

39.9% 

1:42:04 

3 

7 

44 

0:00:48 

1 

2:40:13 

62.6% 

22.7% 

0:58:09 

1 

4 

28 

0:00:59 

7 

3:15:08 

76.2% 

13.6% 

0:34:55 

7 

1 

22 

0:02:39 

4 

3:42:51 

87.1% 

10.8% 

0:27:43 

4 

3 

10 

0:10:12 

9 

3:58:57 

93.4% 

6.3% 

0:16:06 

9 

9 

4 

0:04:01 

2 

3:58:57 

93.4% 

0.0% 

0:00:00 

2 

2 

0 

5 

3:58:57 

93.4% 

0.0% 

0:00:00 

5 

5 

0 

6 

3:58:57 

93.4% 

0.0% 

0:00:00 

6 

6 

0 

8 

3:58:57 

93.4% 

0.0% 

0:00:00 

8 

8 

0 

10 

3:58:57 

93.4% 

0.0% 

0:00:00 

10 

10 

0 

Table  26.  MYMC  POST:  Details  by  Process. 
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PRE  vs  POST 

PID 

n/PRE 

n/POST 

p-value 

Change  of  the  meanP0ST  relative 

to  that  of  the  PRE 

PIDs, 

l 

35 

22 

0.001 

Increase 

individually 

4 

16 

28 

0.03 

Decrease 

BCMA/eMAR 

1 

(subset  of) 

24 

12 

<  0.0005 

Increase 

Intra-location  geography 

Not  Applicable 

AIDs,  individually 

No  statistical  significance  was  found. 

Table  27.  Summary  of  Statistical  Findings  -  MYMC. 
MIMC 


The  time  and  motion  POST  data  collection  occurred  July  2014  at  MIMC.  This  data  was  compared  to  the 
observation  in  the  PRE  period  (May  2012)  with  the  following  results.  So  as  to  be  directly  comparable,  the 
data  for  each  time  period  was  normalized.  ANOVA  was  applied  to  each  category  over  time  period.  Only 
the  Med  administration  (N,pre  =  72;  N,post  =  128)  yielded  statistical  significance,  p-value  =  .024. 
Homogeneity  of  variance  held  but  normality  was  violated,  therefore  Mann-Whitney  U  was  applied  which 
yielded  a  p-value  =  .004.  Even  with  appropriate  application  of  correction  for  multiple  tests,  the  reduction 
in  time  per  medication  administration  event  from  PRE  to  POST  is  likely  statistically  significant.  Of  note, 
statistical  comparison  for  some  categories  was  not  calculable  due  to  very  small  (even  zero)  sample  size, 
but  nonetheless  likely  represent  a  meaningful  change.  Testing  by  individual  activity  was  hampered  by 
normality  violations  and  both  small  and  severely  disparate  sample  sizes.  Even  for  the  few  activities  for 
which  the  Mann-Whitney  test  was  feasible,  no  statistical  significance  was  found.  Please  refer  to  Figure 
35  and  Table  28  and  Table  29  for  details. 


Nurse  Shadowing:  Summary  of  Time,  percentage  basis 
Location  =  MINERS 

HPRF  BPC8T  -Hfr— ftiffi  J  5tiv£  %.  PRC  — *—  GmW  ul3t*v$  <Hj_  POST 


(FID):  Description 


Figure  35.  Nurse  Shadowing:  Summary  of  Percentages,  MIMC. 
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MIMC  PRE  (N  =  1  observation  -  RN): 


Nurse  Shadowing :  Study  time  Period^  -  PRE,  Location  -  MiMC 


Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cummulative 

time, 

hh:mm:ss 

cummulative 
time  per  total 
time,  % 

%  of  Total 
Time  by 
PID 

Total  Time  / 
PID, 

hh:mm:ss 

PID 

PID 

Count 

of 

Activity 

IDs 

avg.  time  / 
AID 

hh:mm:ss 

1 

1:59:02 

49.5% 

49.5% 

1:59:02 

1 

1 

72 

0:01:19 

3 

2:29:40 

62.2% 

12.7% 

0:30:38 

3 

7 

44 

0:03:36 

7 

2:54:12 

72.4% 

10.2% 

0:24:32 

7 

4 

32 

0:00:36 

10 

3:18:24 

82.5% 

10.1% 

0:24:12 

10 

11 

16 

0:00:45 

4 

3:37:06 

90.3% 

7.8% 

0:18:42 

4 

3 

9 

0:03:24 

11 

3:49:13 

95.3% 

5.0% 

0:12:07 

11 

2 

3 

0:02:01 

2 

3:55:17 

97.8% 

2.5% 

0:06:04 

2 

8 

2 

0:01:30 

8 

3:58:18 

99.1% 

1.3% 

0:03:01 

8 

9 

2 

0:00:51 

9 

3:59:59 

99.8% 

0.7% 

0:01:41 

9 

10 

2 

0:12:06 

6 

4:00:32 

100.0% 

0.2% 

0:00:33 

6 

6 

1 

0:00:33 

5 

4:00:32 

100.0% 

0.0% 

0:00:00 

5 

5 

0 

Table  28.  MIMC  PRE:  Details  by  Process. 
MIMC  POST  (N  =  1  observation  -  RN): 


Nurse  Shadowing :  Study  time  Period^  -  POST,  Location  -  MiMC 


Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cummulative 

time, 

hh:mm:ss 

cummulative 
time  per  total 
time,  % 

%  of  Total 
Time  by 
PID 

Total  Time  / 
PID, 

hhimmiss 

PID 

PID 

Count 

of 

Activity 

IDs 

avg.  time  / 
AID 

hh:mm:ss 

1 

2:36:50 

66.1% 

66.1% 

2:36:50 

1 

1 

128 

7 

3:17:19 

83.2% 

17.1% 

0:40:29 

7 

7 

43 

0:00:34 

4 

3:31:28 

89.2% 

6.0% 

0:14:09 

4 

4 

17 

0:00:45 

11 

3:44:45 

94.8% 

5.6% 

0:13:17 

11 

11 

12 

0:02:31 

8 

3:51:11 

97.5% 

2.7% 

0:06:26 

8 

3 

4 

0:01:19 

3 

3:56:29 

99.7% 

2.2% 

0:05:18 

3 

8 

2 

0:03:13 

9 

3:57:07 

100.0% 

0.3% 

0:00:38 

9 

9 

1 

0:00:38 

2 

3:57:07 

100.0% 

0.0% 

0:00:00 

2 

2 

0 

5 

3:57:07 

100.0% 

0.0% 

0:00:00 

5 

5 

0 

6 

3:57:07 

100.0% 

0.0% 

0:00:00 

6 

6 

0 

10 

3:57:07 

100.0% 

0.0% 

0:00:00 

10 

10 

0 

Table  29.  MIMC  POST:  Details  by  Process. 
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Nursing  Time  and  Motion  Summary 


Similar  to  other  academic  medical  centers,  BCMA  was  not  associated  with  an  increase  in  the  amount  of 
time  spent  on  medication  administration-related  activities  (Poon  EG  2008  Dec).  Small  differences  were 
noted  on  specific  clinical  units,  and  medication  administration  efficiencies  could  therefore  be  somewhat 
affected  by  patient  acuity.  Significant  time  differentials  were  not  specifically  noted. 

AHRQ  hospitals  reported  no  long-term  impacts  on  nurse  efficiency.  Though  BCMA  does  not  appear  to 
increase  workload,  realistic  expectation  should  be  set  with  nursing  staff  that  it  is  not  a  time-saving 
technology  (Hook  JM  2008).  Patient  safety  effects  should  be  self-evident,  but  BCMA  work  arounds 
continually  evolve. 

Overall  Workflow  Summary  Indications 

In  agreement  with  the  Agency  for  Healthcare  Research  and  Quality's  (AHRQ)  (Agency  for  Healthcare 
Research  &  Quality  n.d.)  discussion  on  communication  and  workflow  regarding  bar  coded  medication 
administration,  implementing  BCMA  required  workflow  modifications  for  not  only  pharmacy,  but  also 
nursing,  and  other  stakeholders.  Changes  in  culture,  attitudes,  and  practice,  were  not  only  a  necessity, 
but  evident  in  both  realizing  and  achieving  the  benefits  of  this  technology. 

Redesigning  the  medication  administration  process  as  part  of  implementing  BCMA  required  increased 
collaboration  and  communication  among  pharmacy  and  nursing  staff.  The  workflow  redesign  forced 
those  involved  to  confront  and  re-define  longstanding  perceptions  surrounding  the  roles  and 
responsibilities  of  each  party  in  the  medication  administration  process.  Differing  perspectives  and 
perceptions  have  likely  changed  before,  during,  and  after  BCMA  implementation  and  will  continue  to 
evolve. 

While  the  implementation  of  BCMA  was  expected  to  impact  patient  safety  but  is  outside  of  the 
discussion  for  workflow,  results  in  terms  of  time  savings  were  not  achieved,  similar  to  results  noted  in 
the  literature.  CHS,  similarly  to  BCMA  findings  reported  by  the  AHRQ,  yielded  no  net  impact  on  the 
efficiency  of  nursing  staff  and  did  possibly  create  additional  work  for  pharmacy  staff. 

While  the  benefits  of  BCMA  are  realized,  human  factors,  along  with  basic  system  functionality  and 
organizational  infrastructure,  need  to  be  given  careful  consideration  before  during  and  after 
implementation.  Moreover,  the  complex  and  dynamic  nature  of  care  processes  and  diverse  types  of 
system  users  can  widen  the  gap  between  the  system-intended  process  and  the  actual  work  process. 
Systems  implemented  without  consideration  of  existing  work  flows  and  user  adaptation  can  provoke 
unfavorable  perceptions  and  reluctance  on  the  systems  and  theses  can  be  major  causes  for  non- 
compliance  behaviors  and  workarounds  (Holden  RJ  2011)  (Yang,  et  al.  2012)  (Miller  2011)  (Halbesleben 
JRB  2010)  (Rack,  Dudjak  and  &  Wolf  2012). 

Nursing  and  pharmacy  staff  cooperation/communication  is  essential  during  both  medication  order 
verification  and  administration.  BCMA  however  demonstrates  an  improved  accountability  and 
measurement  of  the  medication  administration  process.  Reporting  capabilities  of  many  BCMA  systems 
support  analysis  of  when  a  medication  was  administered,  who  administered  it,  and  whether  medication 
information  was  scanned  or  manually  entered  into  the  system.  With  many  hospitals,  Conemaugh 
included,  bar  coding  benefits  at  all  points  of  medication  distribution  may  be  further  realized  as 
healthcare  continues  to  transform  and  redefine  itself  (Wideman,  ME  and  Anderson  2005). 


62 


1.2.11  Medication  Error  Analysis 

Lewis  et  al.  (Lewis  PJ  2009)  found  that  an  average  medication  error  rate  in  all  prescribing  orders  is 
estimated  to  7%.  It  is  also  estimated  that  every  two  admissions  experience  a  medication  error.  Thirty- 
eight  percent  of  preventable  medication  errors  however,  occurred  at  the  medication  administration 
process  (Shane  2009).  With  such  statistics,  the  overall  impact  of  the  BCMA  system  on  the  organization 
would  not  be  complete  without  a  medication  error  analysis. 

Medication  Errors  Type 

The  research  team  completed  intensive  review  of  all  CHS  medication  events  as  reported  in  the  Quantros 
SRM  event/incident  reporting  system.  Medication  events  were  reviewed  at  all  CHS  facilities  (MMC, 
MYMC  and  MIMC).  All  errors  that  occurred  within  a  unit/department  that  had  not  deployed  BCMA 
technology  were  removed  from  the  dataset. 

Errors  were  examined  via  disposition,  applicability  to  MIDHT,  phase,  and  core  problem  (medication  error) 
type.  Level  of  analysis  are  described  below  and  detailed  in  Table  30. 

•  Error  Disposition:  This  designation  was  completed  by  Quantros  SRM  users  and  qualifies  as  an 
"actual"  or  "near  miss." 

•  Applicability  to  MIDHT:  The  research  team  used  both  medication  error  type  and  phase  to 
subjectively  determine  MIDHT  Applicability  (as  designated  as  "Yes"  or  "No")  for  each  error.  In 
other  words,  based  on  the  reported  medication  event,  could  the  error  be  attributed  to  the  BCMA 
technology? 

•  Phase:  The  phase  was  subjectively  determined  by  the  study  team  using  common  industry 
standards  detailing  the  medication  administration  process.  Phase  identification  was  consistent 
with  current  literature.  The  MIDHT  intervention  primarily  consisted  of  technology  and  process 
changes  that  only  impacted  the  Dispense  and  Administration  phases  through  pharmacy  robotics 
and  BCMA  at  the  bedside. 

•  Core  Problem  (Medication  Error)  Type:  This  was  predefined  within  Quantros  using  standardized 
phrases.  Some  of  the  errors  were  scrutinized  in  an  attempt  to  determine  the  origination  of  the 
issue.  A  small  percentage  of  these  standard  error  types  were  aggregated  by  the  study  team  to 
enable  statistical  testing  on  this  variable  (Table  31  below). 

The  influence  of  the  MIDHT  intervention  on  medication  errors  from  before  implementation,  PRE,  versus 
after  implementation,  POST,  was  investigated  using  the  Pearson  Chi-squared  test.  Fisher's  Exact  Test 
was  employed  as  required,  e.g.  when  the  count  of  any  cell  in  the  crosstab  being  analyzed  was  less  than 
5.  The  ordinal  variable.  Study  Time  Period,  was  treated  as  the  independent  variable  (IV).  It  has  only  two 
levels,  PRE  and  POST,  both  which  contained  12  months  of  data  for  each  clinical  unit  (MMC)  or  site.  The 
statistical  analysis  plan  was  comprised  of  two  major  steps: 

1.  Each  of  the  following  variables  were  treated  as  dependent  variables  and  separately  tested 
against  Study  Time  Period.  The  dependent  variables  are  as  follows: 
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Variable  Name 

Variable  Type 

Levels 

Error  Disposition 

nominal 

Near  Miss 

Actual 

MIDHT  Applicability 

nominal 

Yes 

No 

PHASE 

ordinal 

ORDER  (1st) 

PROFILE  (2nd) 

DISPENSE  (3rd)  + 

ADMINISTRATION  (4th)  + 

Other 

Core  Problem  (Med. 
Error)  Type 

nominal 

Please  see  Table  31  for  a  detailed 

listing 

Table  30.  Variable  Name  Description.  +  Primary  MIDHT  intervention  focus 

2.  If  statistical  significance  was  found  for  a  test  variable  in  step  1,  a  layered  analysis  by  MIDHT 
Applicability  on  that  variable(s)  over  Study  Time  Period  was  then  executed.  This  approach  allows 
for  a  more  detailed  investigation  of  change  from  PRE  to  POST  by  using  MIDHT  Applicability  to 
stratify  the  variable  being  tested  in  step  2.  Please  note  these  original  medication  error 
designations  were  later  regrouped  based  upon  similarities  between  the  error  types. 


Core 

Core  Problem  (Med  Error)  Type 

Problem 

Code 

Bar  Coding/Scanning  Error 

1 

Charting  Error 

2 

Drug  Delay 

3 

Drug  Location  Issue 

4 

Drug  Not  Ordered 

5 

Drug  Omitted 

6 

Drug  Protocol/Policy  Not  Followed 

7 

Incorrectly  Stamped  Order  Sheet 

8 

IV  Related  Issue 

9 

Medication  Reconcilliation  Issue 

10 

Order  Issue 

11 

Order  Issue/Not  Entered 

12 

Patient  Not  Compliant 

13 

Pharmacy  Profile  Error 

14 

Pyxis  Issue 

15 

Wrong  Dose 

16 

Wrong  Drug 

17 

Wrong  Frequency 

18 

Wrong  Label 

19 

Wrong  Patient 

20 

Wrong  Quantity 

21 

Wrong  Route 

22 

Table  31.  Medication  Error  Type  Listing. 
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The  medication  administration  process  is  not  simply  the  act  of  administering  medication,  but  rather 
complex  processes  involving  multiple  steps  from  numerous  disciplines,  departments,  and  users.  Errors 
can  appear  at  one,  some  or  even  all  stages  between  the  medication  order  process  (prescription)  through 
the  actual  administration  of  the  medication  and/or  possible  adverse  events.  Though  a  specific  error  may 
occur  prior  to  a  patient  receiving  the  medication,  most  errors  are  noted  only  once  the  medication  is 
incorrectly  administered  at  the  bedside. 

Medication  Errors  by  Type  -  MMC 

Researchers  reviewed  over  1,100  written  incident  summaries  (November  2010  -  January  2013)  in  detail 
and  logically  determined  the  impact  on  the  MIDHT  study  using  the  above  mentioned  variables.  The  PRE- 
BCMA  time  period  for  MMC  was  November  2010  -  October  2011.  The  POST-BCMA  time  period  was 
from  November  2011  -January  2013.  An  additional  three  months  were  included  in  the  POST-BCMA 
period  because  of  the  phased  BCMA  deployment  schedule  at  Conemaugh.  The  final  dataset  contained 
630  valid  cases. 


Common  errors,  such  as  wrong  patient,  wrong  drug,  and  wrong  dose  all  had  a  somewhat  neutral  change 
from  PRE  to  POST  (Figure  36).  The  low  sample  size  per  error  type  was  an  issue  and  produced  many 
violations  of  the  allowable  percentage  of  cells  with  a  minimum  expected  frequency  less  than  five. 
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Pyxi$  Issue 
Drug  Location  Issue 
Wrong  Quanti 
Bar  Codling/Scanning  Erro 
Wrong  Labe 
Drug  Not  Orders 
Patient  Not  Compliant 
Wrong  Route 
Drug  Protocol/Policy  NotFollOwe 
Medication  Reconciliation  Issue 
Charting  Erro 
Pharmacy  Profile  Erro 


IV  Related  Issue 
Wrong  Frequenc 
Wrong  P alien' 
Order  issue 
Wrong  Dru 
Wrong  Dose- 
Incorrectly  Stamped  Order  Shee' 
Drug  Dela; 
Drug  Omitte 


Study 

Time 

Period 

■  pRE 

■  POST 


Count 


Figure  36.  Core  Problem  Type  Counts  by  Period  -  MMC. 

The  results  of  the  stepwise  analysis  at  MMC,  which  tested  each  of  the  dependent  variables  separately, 
were  as  follows: 
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Crosstab 

Chi- 

squared, 

df 

Sample 

size,  N 

p-value, 

asymp. 

Sig.  (2- 

Error  Disposition  *  Study  Time  Period 

No  statistical  significance. 

MIDHT  Applicable  *  Study  Time  Period 

No  statistical  significance. 

PHASE  *  Study  Time  Period 

26.386 

4 

630 

<  .0005 

Core  Problem  (Error)  Type  *  Study  Time  Period 

25.489 

11 

630 

.008  a 

2  cells  (8.3%)  have  expected  count  <  5.  Min.  expected  count 

is  1.93. 

Table  32.  Crosstab  Inferential  Results. 

Sidak's  correction  for  multiple  testing  was  applied  which  reduced  the  testwise  alpha  criterion  to  0.017. 
This  correction  was  slightly  conservative  in  that  no  correlation  was  assumed  to  exist  among  the 
variables.  A  correlation  analysis  of  (between)  the  dependent  variables  revealed  an  average  correlation 
of  0.1,  which  rises  slightly  to  0.3  if  only  the  statistically  significant  correlations  are  averaged.  Including 
the  0.3  correlation  into  the  calculation  of  Sidak's  correction  yields  a  testwise  alpha  criterion  to  0.019  and 
corresponding  critical  z-value  for  2  sided  testing:  >=  2.3407  (critical  z-value  for  1  sided  testing:  >= 
2.0695).  Therefore,  only  Phase  *  Study  Time  Period  and  Core  Problem  (Error)  Type  *  Study  Time  Period 
crosstabs  had  a  statistically  significant  result.  A  progression  of  the  original  designation  of  medication 
errors  type  can  be  found  below  in  Table  33. 


Core  Problem  (Med  Error)  Type  -  Original 

Bar  Coding/Scanning  Error 

Charting  Error 

Drug  Delay 

Drug  Location  Issue 

Drug  Not  Ordered 

Drug  Omitted 

Drug  Protocol/Policy  Not  Followed 

Duplicate  Therapy 

Incorrectly  Stamped  Order  Sheet 

IV  Related  Issue 

Medication  Reconciliation  Issue 

Order  Issue 

Patient  Not  Compliant 

Pharmacy  Profile  Error 

Pyxis  Issue 

Wrong  Dose 

Wrong  Drug 

Wrong  Frequency 

Wrong  Label 

Wrong  Patient 

Wrong  Quantity 

Wrong  Route 

Core  Problem  (Med  Error)  Type  -  Revised 

Bar  Coding/Scanning  Error 

Charting  Error 

Drug  Delay 

Drug  Location  Issue 

Drug  Omitted 

Duplicate  Therapy 

Wrong  Dose 

Wrong  Drug 

Wrong  Frequency 

Wrong  Label 

Wrong  Patient 

Wrong  Quantity 

Wrong  Route 

Table  33.  Progression  of  Medication  Error  Type  Listing. 


66 


These  results  indicated  that  Phase  and  Study  Time  Period  are  not  statistically  independent  over  the 
entire  dataset  of  medication  errors;  therefore  the  differences  in  the  distribution  by  phase  by  Study  Time 
Period  are  statistically  significant.  Standardized  residuals  of  the  crosstab  revealed  that  Profile  for  PRE 
and  POST  mainly  contributed  to  the  significant  chi-square  result.  Collectively,  this  implies  that  the 
observed  increase  from  PRE  to  POST  is  significant.  Given  the  statistically  significant  result  for  the 
crosstab  of  Phase  *  Study  Time  Period,  a  more  detailed  analysis  was  performed  on  a  stratified  crosstab 
of  Phase  by  MIDHT  Applicability  versus  Study  Time  Period.  Statistical  significance  was  found  for  only 
MIDHT  Applicability  =  NO  for  the  distribution  of  Phase  between  Study  Time  Period  [chi-sq.(4)  =  29.959, 
p-value  =  <  .0005]. 

The  results  indicate  that  the  distribution  of  Core  Problem  (Error)  Type  differs  over  Study  Time  Period. 
The  standardized  residuals  imply  the  following  variables  in  Table  34  contribute  to  that  difference: 


Implication  regarding  change 

from  PRE  to  POST 

Drug  Location  Issue 

decrease 

Wrong  Frequency 

increase 

Note  that  the  implications  drawn  from  the  Core  Problem  (Error)  Type  *  Study  Time 
Period  chi-squared  analysis  is  made  regarding  the  distribution  of  errors  by  error  type  by 
time  period. 


Table  34.  Standardized  Residual  Results. 

Given  the  statistically  significant  result  for  the  crosstab  of  Core  Problem  (Error)  Type  *  Study  Time 
Period,  a  more  detailed  analysis  was  performed  on  a  stratified  crosstab  of  Core  Problem  (Error)  Type  by 
MIDHT  Applicability  versus  Study  Time  Period.  Statistical  significance  was  found  for  only  MIDHT 
Applicability  =  YES  for  the  distribution  of  error  type  by  Study  Time  Period;  however,  this  result  is  invalid 
because  nearly  40%  of  the  cells  have  an  expected  count  of  <  5.  The  threshold  for  validity  is  <  20%. 

Either  the  frequency  of  each  medication  error  type  was  so  low  that  no  significant  change  was  detected 
or  the  error  can  be  attributed  to  human  error  and  therefore  not  affected  between  time  periods.  Despite 
the  lack  of  statistical  significance,  about  60%  of  the  error  types  showed  a  15%  or  greater  percent  change 
decrease  in  the  POST-BCMA  period.  Medication  error  types,  drug  delay,  wrong  frequency,  wrong  label, 
and  wrong  patient  each  had  over  a  30%  reduction  in  errors  and  therefore  an  assumed  positive  impact 
on  patient  safety. 

Medication  Errors  Analysis  -  MMC 

The  distribution  of  the  severity  of  drug  errors  was  also  analyzed  over  time  period.  The  chi-square 
analysis  produced  an  invalid  statistically  significant  result  because  nearly  40%  of  the  cells  had  an 
expected  count  <  5.  Harm  score  was  analyzed  overtime  period.  Although  no  statistical  significance  was 
found,  this  analysis  was  invalid  in  that  exactly  40%  of  the  cells  had  an  expected  count  <  5.  Importantly, 
90.5%  PRE  and  94.0%  POST  of  the  med  errors  were  marked  with  a  Harm  Score  of  NONE. 


Initial  discussions  were  rooted  in  the  possibility  that  reporting  frequency  of  actual  vs.  near  miss  may 
differ  (e.g.  near  miss  may  be  reported  less  frequently  to  save  time;  or  near  miss  errors  may  be  reported 
more  POST  BCMA  implementation  because  nurses  were  disgruntled/dissatisfied  with  the  technology 
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and/or  disruptions  to  their  traditional  workflow,  PRE  implementation).  Some  studies  have  reported  a 
nearly  50%  reduction  in  medication  errors  post  BCMA  implementation. 

The  distribution  of  the  severity  of  drug  errors  was  thus  analyzed  over  time  period. 

It  should  be  noted  however,  the  culture  of  reporting  errors  had  changed  in  the  POST  period,  which 
complicated  the  analysis.  Errors  were  likely  reported  out  of  frustration  with  the  deployed  BCMA  system 
and  discord  between  the  nursing  and  pharmacy  departments. 

The  chi-square  analysis  produced  an  invalid  statistically  significant  result  because  nearly  40%  of  the  cells 
had  an  expected  count  <  5.  Harm  score  was  analyzed  over  time  period.  Although  no  statistical 
significance  was  found,  this  analysis  was  invalid  in  that  exactly  40%  of  the  cells  had  an  expected  count  < 
5.  Importantly,  90.5%  PRE  and  94.0%  POST  of  the  med  errors  were  marked  with  a  Harm  Score  of  NONE. 

Initial  discussions  were  rooted  in  the  possibility  that  reporting  frequency  of  actual  vs.  near  miss  may 
differ  (e.g.  near  miss  may  be  reported  less  frequently  to  save  time;  or  near  miss  errors  may  be  reported 
more  POST  BCMA  implementation  because  nurses  were  disgruntled/dissatisfied  with  the  technology 
and/or  disruptions  to  their  traditional  workflow,  PRE  implementation). 

Some  studies  have  reported  a  nearly  50%  reduction  in  medication  errors  post  BCMA  implementation  (K. 
C.  Poon  EG  2010)  (Franklin  BD  2007)  (Morris  FH  2009).  The  research  team  will  review  these  studies  in 
detail  and  attempt  to  make  a  one-to-one  comparison  between  their  experiences  and  ours  to  generate 
additional  lessons  learned. 

The  medication  error  rate  per  1,000  doses  was  analyzed.  Some  inherent  error  exists  in  this  analysis 
because  the  available  dosage  data  was  not  stratified  by  clinical  unit;  therefore,  matching  of  dosage  data 
with  medication  error  data  by  clinical  unit  that  implemented  BCMA  was  not  possible.  Presuming  that 
this  error  is  consistent  from  PRE  to  POST,  this  comparison  offered  additional  insight  into  any  change 
influenced  by  the  MIDHT  intervention.  A  two-group  independent  T-test  revealed  no  statistical  significant 
difference  in  the  slight  increase  of  the  mean  of  the  med  error  rate  PRE  (0.08)  versus  POST  (0.09). 

PRE:  MED  ERROR  RATE  per  1000  doses 


M  ED_ERft_RATE_perl0OO_dose5 


Figure  37.  MMC  Medication  Error  Rate  per  1,000  doses  -  PRE. 
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POST:  MED  ERROR  RATE  per  1000  doses 
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Figure  38.  MMC  Medication  Error  Rate  per  1,000  does  -  POST. 

Please  note  the  timeframe  plotted  in  Figure  37  and  Figure  38  were  adjusted  to  accommodate  the 
phased  implementation  so  that  all  clinical  units  are  equally  represented  by  time. 

Medication  errors  can  vary  depending  on  the  differing  levels  of  care  per  clinical  area.  Differences 
between  nursing  units  must  be  considered  because  of  nurse-to-patient  ratio,  as  well  as  hospital  census 
differentials.  An  analysis  of  the  distribution  of  medication  errors  by  clinical  unit  was  invalid  due  to  the 
large  number  of  clinical  units  involved,  which  produced  expected  cell  count  issues.  A  recent  literature 
review  study  reported  average  baseline  error  frequency  rates  from  5.8-25.3%  (1.6-27.8%  if  error  times 
were  excluded). 1  Wrong  time  errors  are  generally  considered  to  be  less  severe.  Typical  reports  after 
BCMA  implementation  include  a  30-50%  reduction  in  medication  errors,  but  only  when  time  errors  are 
excluded. 


Work-arounds  performed  by  nursing  staff  must  be  acknowledged  as  a  contributing  factor  on  the 
medication  error  rates.  Table  35  demonstrates  manual  override  statistics  from  MMC  for  July  through 
September  2013.  Though  these  reporting  statistics  were  not  available  during  the  study  time  period, 
inferences  to  the  impact  on  medication  errors  by  clinical  unit  can  still  be  made. 


McK  Adoption  HARx  Analytics  Medication  Administration 

Schedule 

Override 

Count 

%  with 

Schedule 

Override 

Drug  Warning 
Override 

Count 

%  with  Drug 
Warning 
Override 

Quantity 

Override 

Count 

%  with 
Quantity 
Override 

83 

2.96 

79 

2.86 

39 

0.62 

Table  35.  MMC  manual  overrides  by  type  (3-month  average). 

A  correlation  analysis  between  the  override  data  above  and  medication  errors  by  clinical  unit  was  not 
tenable  due  to  violations  arising  from  sample  size  after  stratification. 
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Medication  error  data  was  retrieved  from  Quantros  for  July  -  September  2013  and  summarized  for  use. 
This  analysis  was  continued  so  as  to  focus  on  clinical  units  cited  in  the  literature,  and  to  determine  if 
high  override  percentages  influenced  medication  error  rates. 

Medication  Errors  by  Type  -  MYMC 

Researchers  reviewed  over  80  written  incident  summaries  (May  2011  -  May  2013)  in  detail  and  logically 
determined  the  impact  on  the  MIDHT  study  using  the  above  mentioned  variables.  The  PRE-BCMA  time 
period  for  MYMC  was  May  2011  -  April  2012.  The  POST-BCMA  time  period  was  from  May  2012-2013. 
The  final  dataset  contained  83  valid  cases. 

The  medication  error  data  for  MYMC  was  analyzed  by  comparing  pre  versus  post  on  the  counts  of  error 
type,  phase,  harm  and  severity  score.  Due  to  the  low  sample  size  at  this  critical  access  hospital,  many  of 
the  chi-square  crosstabs  violated  the  allowable  percentage  of  cells  with  a  minimum  expected  frequency 
less  than  five.  One  crosstab,  phase  by  time  period  did  not  violate  the  requirement,  but  nonetheless,  did 
not  show  a  statistically  significant  chi-square  result.  Despite  the  lack  of  statistical  significance,  forty-five 
percent  of  the  identified  error  types  showed  about  a  10%  or  greater  percent  change  decrease  in  the 
post  period  as  displayed  in  Figure  39.  Both  duplicate  therapy  and  wrong  dose  had  a  clinically  significant 
reduction  and  therefore  positive  impact  on  patient  safety. 
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Figure  39.  Core  Problem  Type  Counts  by  Period  -  MYMC. 
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For  example,  wrong  dose  had  a  clinically  significant  reduction  and  positive  impact  on  patient  safety. 
Error  types  with  blanks  are  not  calculable  because  no  data  was  present  in  the  pre  time  period. 
Frequency  is  the  difference  in  count  between  time  periods. 

The  mean,  median,  and  distribution  of  count  of  medication  errors,  doses,  and  medication  error  rate 
were  tested  using  a  Studentized-t,  Median,  and  Mann-Whitney  U  tests  respectively.  Normality  and 
homogeneity  of  variance  held.  No  statistical  significance  was  revealed.  Medication  error  rates  are 
shown  below  in  Figure  40  and  Figure  41: 


MYMC:  MED  ERROR  RATE  per  1,000  doses 
-♦-Pre 

1.0  - 

0.9  -U 


Figure  40.  MYMC  Med  Error  Rate  per  1,000  doses  -  PRE. 


MYMC:  MED  ERROR  RATE  per  1,000  doses 
—♦—Post 


Figure  41.  MYMC  Med  Error  Rate  per  1,000  doses  -  POST. 
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Medication  Errors  by  Type  -  MIMC 


Researchers  reviewed  over  200  written  incident  summaries  (May  2011  -  May  2013)  in  detail  and 
logically  determined  the  impact  on  the  MIDHT  study  using  the  above  mentioned  variables.  The  PRE- 
BCMA  time  period  for  MIMC  was  March  2013  -  February  2014.  The  POST-BCMA  time  period  was  from 
March  2014  -  February  2015.  The  final  dataset  contained  192  valid  cases.  Since  the  amount  of 
medication  errors  was  expected  to  be  low,  an  entire  year  period  is  warranted  for  proper  statistical 
analysis. 

The  dose  count,  med  error  count,  and  med  error  rate  per  1,000  doses  were  compared  over  time  period. 
Data  was  time-matched  on  month  (N=12  per  time  period).  Normality,  albeit  with  some  violation,  and 
homoscedasticity  with  the  exception  of  dose  count  held  for  all  three  variables  for  both  pre  and  post 
time  periods  per  Shapiro-Wilk  and  Levene's  Tests  respectively.  A  t-test  revealed  no  statistically 
significant  difference  of  means,  p  =  .201,  .125,  .065  respectively. 

Due  to  the  observed  normality  violations,  Mann-Whitney  U  on  distribution  and  the  Median  Tests  were 
also  applied  with  results  consistent  with  those  of  the  more  powerful  t-test. 

The  Chi-squared  test  was  utilized  to  test  the  change  in  med  errors  on  time  period  (PRE,  n  =  113  vs.  POST, 
n  =  78)  by  MIDHT  applicability  (yes,  no).  Phase  (order,  profile,  dispense,  administration,  other),  and 
actual  medication  error  or  near  miss.  Only  actual/near  miss  showed  a  statistically  significant  change,  p  < 
.0005;  furthermore,  the  percentage  within  time  period  was  also  statistically  significant  on  each  level  of 
actual/near  miss.  The  standardized  residuals  indicate  that  more  of  the  statistical  significance  lies  with 
the  near  miss  category,  which  decreased  from  37%  to  12%  within  time  period  while  the  percentage  of 
actual  medication  errors  increased  from  63%  to  89%. 
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Figure  42.  Core  Problem  Type  Counts  by  Period  -  MIMC. 
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POST:  MED  ERROR  RATE  per  1000  doses 


Mod_E  rror_R,a  te_per  1 0GQ_d  oses 


Figure  43.  MIMC  Med  error  Rate  per  1,000  doses  -  PRE. 


PRE:  MED  ERROR  RATE  per  1000  doses 

— Mod_£  rror_Ra  te_per  1 000_doses 


Figure  44.  MIMC  Med  Error  Rate  per  1,000  doses  -  POST. 
General  Discussion  -  Medication  Errors 


The  medication  administration  process  is  not  simply  the  act  of  administering  medication,  but  rather 
complex  processes  involving  multiple  steps  from  numerous  disciplines,  departments,  and  users.  Errors 
can  appear  at  one,  some  or  even  all  stages  between  the  medication  order  process  (prescription)  through 
the  actual  administration  of  the  medication  and/or  possible  adverse  events.  Though  a  specific  error  may 
occur  prior  to  a  patient  receiving  the  medication,  most  errors  are  noted  only  once  the  medication  is 
incorrectly  administered  at  the  bedside. 

Despite  the  fact  that  BCMA  strives  to  assure  the  "five  rights"  of  medication  administration  -  right 
patient,  right  route,  right  drug,  right  dose,  and  right  time,  most  studies  however  have  investigated  the 
effect  of  BCMA  on  the  rate  of  severity  of  the  errors  or  the  effect  of  the  BCMA  on  the  duration  of 
administering  medication. 
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Medication  errors  can  vary  depending  on  the  differing  levels  of  care  per  clinical  unit.  A  recent  literature 
review  study  (Hassink  J  2012)  reports  average  baseline  error  frequency  rates  from  5.8-25.3%  (1.6-27.8% 
if  error  times  were  excluded).  Wrong  time  errors  are  generally  considered  to  be  less  severe.  Typical 
reports  after  BCMA  implementation  include  a  30-50%  reduction  in  medication  errors,  but  only  when 
time  errors  are  excluded. 

The  severity  of  drug  errors  should  also  be  included  in  discussion.  Some  studies  have  reported  a  nearly 
50%  reduction  in  risks  post  BCMA  implementation  (K.  C.  Poon  EG  2010)  (Franklin  BD  2007)  (Morris  FH 
2009).  Error  disposition  (i.e.  near  miss  or  actual  miss)  per  time  periods,  nursing  unit,  and  other 
appropriate  delineations  may  lead  to  subsequent  conclusions  about  BCMA  implementation.  Initial 
discussions  were  rooted  in  the  possibility  that  reporting  frequency  of  actual  vs.  near  miss  may  differ  (e.g. 
near  miss  may  be  reported  less  frequently  to  save  time;  or  near  miss  errors  may  be  reported  more  POST 
BCMA  implementation  because  nurses  were  disgruntled/dissatisfied  with  the  technology  and/or 
disruptions  to  their  traditional  workflow,  PRE  implementation). 

Most  medication  error  research  however  has  been  completed  in  small  units  and  thus  for  comparative 
purposes  that  final  data  analyses  may  examine  the  impact  of  BCMA  on  specifically  identified  nursing 
units  at  Conemaugh.  Differences  between  nursing  units  must  be  considered  because  of  nurse-to- 
patient  ratio,  as  well  as  hospital  census  differentials. 

Work-arounds  performed  by  nursing  staff  must  be  acknowledged  as  a  contributing  factor  on  the 
medication  error  rates.  Table  35  demonstrates  manual  override  statistics  from  MMC  for  July  through 
September  2013.  Though  these  reporting  statistics  were  not  available  during  the  study  time  period, 
researchers  utilized  the  data  to  make  inferences  to  the  impact  on  medication  errors  by  clinical  unit. 

Considerations  for  future  research  are  to  see  if  medication  errors  could  be  segregated  based  on 
origination  (i.e  if  drug  dispensed  from  the  robot  vs  the  MedCarousel  vs.  manual  picks  results  in  any 
differences  in  the  number,  type  etc.  of  medication  errors).  No  past  studies  investigated  user  compliance 
in  the  BCMA  system  (ie.  work-arounds  with  the  BCMA  system).  Such  may  need  to  be  re-evaluated  over 
time  since  as  system  familiarity  evolves,  so  could  the  work-arounds.  Longer-term  follow-up  studies  (>  2 
years)  are  needed  both  for  the  aforementioned  reason  and  also  for  examination  of  the  degree  and  time 
post  implementation. 

1.2.12Press  Ganey™  surveys  -  MMC  (Patient  satisfaction) 

Press  Ganey™  surveys  distributed  as  part  of  standard  hospital  operations  were  assessed  to  evaluate 
patient  satisfaction  in  regards  to  BCMA.  Data  collection  time  periods  ran  successively  from  November 
2010  -  January  2012  (PRE)  and  from  November  2011  -  January  2013  (POST).  Over  4,000  surveys  were 
reviewed  by  members  of  the  research  team.  Forty-nine  medication  related  comments  were  retrieved 
during  the  PRE  period  and  63  during  the  POST  data  collections  phase  (Table  36).  All  medication-related 
comments  and  associated  groupings  were  reviewed  by  the  BCMA  Clinical  Team  Leader  and  the  Patient 
Representative  from  Patient  Relations  at  MMC.  Neither  had  recommendations  or  comments;  accepting 
the  data  as  presented.  It  is  important  to  note,  however,  that  many  pain  medications  (controlled 
substances)  are  dispensed  from  Pyxis  machines  and  are  not  a  part  of  the  pharmacy  automation 
processes.  MIMC  and  MYMC  used  different  survey  instruments. 
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Comment  Type 

PRE 

Count 

POST 

Count 

Delay 

13 

25 

Medication  administration  issue 

8 

8 

Did  not  receive  medication 

8 

9 

Adverse  reaction/allergy 

8 

10 

Medication  order/rec  issues 

5 

8 

Lack  of  info  about  med  provided  to  pt 

3 

N/A 

Other 

2 

1 

Positive  med  comments 

2 

N/A 

Positive  BCMA  opinion 

N/A 

1 

IT  issue 

N/A 

1 

TOTAL  surveys 

1,989 

2,066 

Table  36.  Press  Ganey  Comments  -  PRE  &  POST. 

A  statistical  analysis  comparing  the  percentage  of  comment  type  over  time  period  was  performed  for 
the  following  comment  types  (Table  37).  A  p-value  <  .05  implies  that  the  change  in  percentage  is 
statistically  significant.  Significance  was  found  for  comments  related  to  medication  delays.  Caution  must 
be  applied  given  that  the  percentages  being  tested  are  extremely  small  relative  to  the  sample  size  (total 
surveys)  and  the  expected  1-tailed  p-value  is  significant,  yet  the  2-tailed  p-value  is  not  significant. 


Comment  Type 

p-value 

l-tail 

2-tail 

Delay 

0.0330 

0.0659 

Adverse  reaction/allergy 

0.3475 

0.6951 

Medication  order/  rec  issues 

0.2221 

0.4443 

Table  37.  p-values  for  Press  Ganey  Patient  Comments  -  PRE  &  POST. 

1.2.13  CHS  Physician  BCMA/eMAR  Survey  (Physician  Satisfaction) 

Forty-two  (42)  clinicians,  physicians,  residents,  hospitalists,  or  physician  assistants  (PAs)  completed  the 
Physician  Survey  (Appendix  G).  Responses  were  collected  from  May  2012  through  March  2013,  POST 
BCMA  implementation.  A  copy  of  the  survey  is  included  as  an  attachment. 

Two  separate  survey  announcements  via  mass  mailing  were  made,  the  second  in  November  2012,  given 
the  small  response  (n=9)  of  the  first  survey  initiative,  conducted  in  May  of  the  same  year.  Of  those 
survey  respondents,  59%  were  physicians,  27%  residents,  2%  hospitalists,  and  12%  PAs.  The  respondents 
indicated  a  diversity  of  years  of  experience,  with  44%  indicating  five  or  less,  22%  twenty  or  more,  and 
the  remaining  34%  from  6-20  years.  73%  did  not  have  BCMA  or  eMAR  experience  outside  of 
Conemaugh.  The  majority  of  those  surveyed  responded  favorably.  Summary  points  from  the  survey  are 
provided  below. 
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•  90%  of  respondents  indicated 
that  they  accessed  the  eMAR 
module  through  CarePortal 


Q1  How  often  do  you  access  the  electronic 
medication  administration  record  (eMAR) 
module  available  through  Care  Portal? 

Answered:  42  Skipped:  0 


Figure  45.  Frequency  of  Access  to  eMAR  module. 


•  67%  either  agreed/strongly 
agreed  that  the  eMAR  module 
has  improved  decision-making 
and  efficiency 


Q2  The  new  eMAR  module  has  improved  my 
decision-making  and  efficiency  because  I 
can  now  electronically  access  if/when 
ordered  medications  were  administered 
from  various  locations. 

Answered:  42  Skipped:  0 


Strongly 

Disagree 


0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


Figure  46.  eMAR  Has  Improved  Decision-Making. 
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•  43%  believed  that  adverse  drug 
events  were  reduced  due  to 
BCMA 


Q3  Adverse  drug  events  have  reduced 
because  of  the  bar  coding  technology  and 
additional  nursing  safeguard 
warnings/alerts  in  place  to  reduce 
medication  errors. 

Answered:  42  Skipped:  0 


Strongly  Agree 


Disagree  4.76% 


Strongly 

Disagree 


0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


Figure  47.  Reduction  in  Adverse  Drug  Events. 


•  24%  believed  that  BCMA 
improved  physician 
communication  with  pharmacy 
and  nursing  (60%  were  unsure) 


Q4  The  new  Bar  Code  Administration 
Administration  (BCMA)  system  and  Care 
Portal  eMAR  module  have  improved 
physician  communication  with  pharmacy 
and  nursing. 

Answered:  42  Skipped:  0 


0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


Figure  48.  Improved  Communication  Between  Pharmacy  and  Nursing. 
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•  48%  would  recommend  BCMA  and  the  accompanying  CarePortal  eMAR  module 


Q5  I  would  recommend  our  new  BCMA 
system  and  Care  Portal  eMAR  module  to 
other  acute  care  hospitals. 

Answered:  42  Skipped:  0 


Disagree  |  4.76% 


Strongly 

Disagree 

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


Figure  49.  Recommend  to  Other  Hospitals. 

Additional  statistical  analyses,  chi-square  crosstabs  by  previous  BCMA  experience  and  years  of  service  and  non- 
parametric  correlation,  were  attempted.  These  analyses  were  not  tenable  due  to  violations  of  required  underlying 
assumptions. 

Survey  results  by  physicians  indicated  the  BCMA  system  was  not  only  favorably  reviewed  (nearing  50%  positivity 
across  categories),  but  also  impacted  efficiencies  and  reduced  adverse  drug  events. 

2.1.14  Late  Medication  Administrations  (Re-budget  Item) 

The  percentage  of  late  medication  administrations  is  reported  as  a  way  to  measure  the  impact  of  the 
patient  ID  band  printers  deployed  at  the  nursing  stations.  Data  is  available  from  HBI  starting  January 
2013.  Therefore  it  is  challenging  to  conduct  a  Pre  vs.  Post  comparison.  For  the  27  months  reported,  the 
percentage  of  "late  admins"  was  variable  and  saw  a  slight  increase  of  1%  over  time.  Late  is  defined  as 
greater  than  one  hour  past  the  ordered  administration  time. 
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Conemaugh  Health  System 
Percentage  of  Late  Medication  Administrations 


Figure  50.  Percentage  of  Late  Medications. 

Lessons  Learned 

Equipment  issues  were  particularly  troublesome  and  created  challenges  for  the  Pharmacy  staff.  These 
issues  included  the  filling  speed  and  location  of  stat  orders,  repackager  inconsistencies  and  applicability 
to  certain  medications,  suction  issues  with  the  robotic  arm  that  resulted  in  dropped  (missing) 
medications,  and  increased  noise  levels.  One  area  of  continued  concern  was  the  envelope  delivery 
system,  or  EDS.  The  EDS  will  sporadically  mis-function  and  cause  some  medications  to  miss  the 
envelope.  The  vendor  has  improved  the  process  via  equipment  adjustments,  but  reliability  issues 
remained. 

Unanticipated  workflow  issues  for  Nursing  came  to  light  as  the  new  process  was  rolled  out.  One  in 
particular  was  the  continuation  of  previous  habits.  The  previous  decentralized  medication  distribution 
model  required  nursing  to  pull  all  patient  medications  from  automated  dispensing  cabinets  (ADC's) 
located  on  each  nurse  station.  During  the  project  planning  phase.  Pharmacy  decided  to  leave  the 
medications  in  the  ADC's  as  a  precaution  in  the  event  the  new  centralized  distribution  process  had 
problems.  Pharmacy  discovered  that  Nursing  was  continuing  to  pull  all  their  medications  from  the  ADC 
instead  of  from  the  envelopes.  The  correct  process  is  to  pull  only  narcotics  from  the  ADC,  since  all  other 
medications  are  to  be  in  the  envelope.  This  resulted  in  envelopes  being  returned  to  the  pharmacy  full  of 
medications  and  requiring  manual  crediting  to  the  patient's  account.  The  issue  was  exacerbated  by  the 
fact  that  the  bar  code  medication  administration  (BCMA)  process  was  not  yet  deployed  and  nursing  did 
not  have  the  new  carts  in  place  to  house  the  medications  sent  from  the  pharmacy.  Pharmacy  addressed 
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the  issue  by  providing  additional  education  to  Nursing  and  by  removing  the  scheduled  meds  from  the 
ADC's  so  that  nursing  must  use  the  envelopes.  Anecdotal  concerns  exist  from  nursing  as  they  need  to 
empty  envelopes  before  placing  specific  medications  into  drawers  on  the  mobile  medication  carts. 

A  second  workflow  issue  involved  hours  of  Pharmacy  operation.  Under  the  decentralized  model,  night- 
shift  was  a  slow  time  with  minimal  activity  other  than  off-hours  order  processing.  This  changed  in  the 
new  Robot-driven  centralized  workflow  model.  Pharmacy  learned  that  they  need  to  utilize  the  midnight 
shift  to  perform  cart  fill  so  that  the  envelopes  for  the  next  day  can  be  delivered  to  Nursing  in  the  early 
morning.  This  change  required  a  review  of  staffing  so  that  additional  staff  is  scheduled  for  the  midnight 
shift. 


A  final  operational  issue  involves  the  capacity  of  the  automation  system.  We  have  learned  that,  even 
with  optimization  efforts,  the  combined  capacity  of  the  Robot  and  MedCarousel  are  not  sufficient  to 
meet  the  needs  of  Conemaugh's  operations.  Conemaugh  contracted  with  McKesson  for  a  second 
MedCarousel  dispensing  system  to  be  installed  in  the  Pharmacy. 


Implementation  of  the  Horizon  Admin  Rx  BCMA  system  has  provided  many  valuable  lessons  to  the 
organization  that  can  be  shared  with  other  stakeholders.  The  implementation  has  resulted  in  a  positive 
safety  improvement  at  all  three  hospital  facilities,  especially  identifying  near-misses  before  an  actual 
error  occurs.  If  the  BCMA  process  is  followed  correctly  by  Nursing,  it  is  highly  likely  the  5  rights  of 
medication  administration  will  be  executed  successfully.  Nursing  likes  the  electronic  medication 
administration  record  (eMAR),  especially  where  they  can  view  past  administrations.  Many  nurses  do 
not  like  paper  now  and  some  of  the  younger  nurses  don't  even  know  the  "old  way."  In  addition, 
physicians  also  have  access  to  the  eMAR  via  Care  Portal  and  they  do  use  this  functionality  when  taking 
care  of  patients.  Transitioning  the  medication  administration  record  from  paper  to  electronic  has  been 
very  beneficial  for  patient  care  and  safety. 


Figure  51.  Admin  RX  eMAR  Module  Screenshot. 


Bar  codes  not  scanning  successfully  have  been  a  chronic  problem  for  the  organization.  Multiple 
barcodes  on  a  single  medication  (e.g.  on  the  medication  and  the  package)  have  resulted  in  the  nurse 
scanning  the  wrong  one.  Multiple  manufacturers  of  the  same  medication  have  also  caused  problems. 
During  the  initial  build  of  the  table,  some  bar  codes  were  missed  by  the  IT  department  and  therefore 
subsequent  scanning  did  not  work.  The  Duke  Lifepoint  acquisition  also  resulted  in  a  major  change  in  the 
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vendors  supplying  medications  to  the  hospital.  Also,  large-volume  IV's  now  come  from  the  Warehouse 
and  these  barcodes  are  sometimes  missed  in  the  table.  Communication  has  improved  between  Nursing 
and  Pharmacy  as  a  result  of  the  most  recent  system  upgrade  which  included  a  messaging  function.  The 
IV  module  is  also  improved  in  this  version,  which  was  a  prior  complaint  from  Nursing.  For  Intermittent 
therapies,  you  have  a  dilutant  plus  a  medication.  Pharmacy  buts  both  items  in  bag  and  creates  one 
barcode  for  both  of  these  items,  the  end-user  is  NOT  to  scan  the  individual  items  but  the  barcode  on  the 
baggies.  MYMC  was  scanning  each  item  separately  and  getting  a  "no  barcode  found  error"  so  they 
stopped  scanning. 

Another  important  lesson  learned  is  to  have  early  buy-in  from  Nursing  and  more  stakeholders  involved 
in  the  training  and  policy  making  phase.  As  we  look  back,  we  cannot  stress  enough  how  important  this 
is  in  creating  a  successful  culture  of  change  within  the  organization.  CHS  still  needs  to  strengthen  its 
culture  of  safety  through  improved  staff  engagement,  ideally  led  by  the  Chief  Medical  Officer  (CMO). 

Conemaugh  purchased  analytical  tools  for  the  Admin  Rx  software  that  provides  very  detailed 
information  (by  user,  unit,  drug,  etc.)  about  the  bar  code  medication  administration  process.  All  nurse 
managers  were  provided  with  direct  access  to  these  reports  via  a  centralized  electronic  system. 
Unfortunately  only  three  (3)  nurse  managers  have  accessed  the  system  as  of  March  2015.  This  result  is 
similar  when  talking  about  "Meaningful  Use"  scorecards  as  well.  Some  issues  have  included 
education/awareness  regarding  the  reports  and  permissions  (security).  Nurse  Managers  have  a  lot  of 
competing  priorities  and  their  use  of  this  information  is  similar  to  other  automated  tools. 
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Arm  2.  Health  Information  Exchange  (HIE)  via  the  Nationwide  Health  Information  Network  (NHIN) 


Subtask  2.1  Deploy  a  limited  production,  NwHIN  standards-based  HIE  focusing  on  the  bi-directional 

exchange  of  electronic  medical  records  between  CHS  and  the  Military  Health  System.  CHS 
information  to  include  data  domains  residing  in  acute  care  and  ambulatory  settings. 

Subtask  2.2  Provide  technical  and  documentation  assistance  on  DoD-managed  Virtual  Lifetime 
Electronic  Record  (VLER)  efforts. 

Subtask  2.3  Investigate  productizing  a  Patient  Consent  module  using  established  standards,  such  as 
TP20/XACML. 

Subtask  2.4  Assess  and  analyze  NwHIN-related  activities,  to  include  data  center  performance  metrics, 

physician  evaluation  and  usage  of  the  NHIN  Portal,  and  resulting  benefits  of  HIE  with  federal 
participants. 

Project  progression  is  briefly  outlined  below  (Figure  52). 
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Figure  52.  MIDHT  Project  Timeline. 

Conemaugh  successfully  completed  all  four  objectives  (subtasks)  with  support  from  partners  that 
included  the:  Telemedicine  and  Advanced  Technology  Research  Center  (TATRC),  Department  of 
Veterans  Affairs,  Northrop  Grumman  Corporation  (NCG)  and  CGI  Federal,  Inc.  Software  development 
activities  are  listed  in  Appendix  H.  All  objectives  were  fulfilled  within  budget  during  the  period  of 
performance. 

When  MIDHT  first  explored  the  use  of  HIE  services,  exchange  was  conducted  through  utilization  of 
Nationwide  Health  Information  Network  exchange  (NwHIN),  formally  a  group  of  stakeholders  and 
integrated  delivery  networks  collaborating  to  securely  exchange  health  information  electronically.  The 
NwHIN  exchange  at  this  point  was  a  departmental  programmatic  operation  of  the  Office  of  the  National 
Coordinator  for  Health  Information  Technology  (ONC).  On  October  12,  2012,  the  ONC  transitioned  to  a 
public-private  partnership  to  better  foster  continued  growth  and  a  sustainable  business  model,  with  the 
NwHIN  now  being  identified  as  "eHealth  Exchange"  (http://www.hitechanswers.net/nwhin-exchange- 
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completes-transition-to-ehealth-exchange/).  The  eHealth  Exchange  (http://sequoiaproiect.org/ehealth- 
exchange/about/history/)  is  made  up  of  over  100  federal  agencies  and  private  partners  that  have 
implemented  national  standards  and  services  for  data  sharing  and  all  parties  execute  the  Data  Use  and 
Reciprocal  Support  Agreement  (DURSA),  a  common  legal  agreement,  in  order  to  securely  exchange 
electronic  health  information  (Figure  53).  Participating  organizations  in  eHealth  Exchange  mutually 
agree  to  support  these  common  set  of  standards  and  specifications  during  the  onboarding  process. 


eHealth  Exchange 


Common  standards,  specifications  and  policies  enforced  through 
Data  Use  &  Reciprocal  Support  Agreement  (DURSA) 


Figure  53.  eHealth  Exchange. 

Conemaugh  was  proud  to  be  the  first  Pennsylvania  non-governmental  health  care  organization  and  23rd 
participant  on  the  eHealth  Exchange.  Production  data  exchange  occurred  with  the  Department  of 
Veterans  Affairs.  Work  allotted  Conemaugh  to  be  recognized  as  the  13th  VLER  pilot  site  in  the  nation. 
Conemaugh  first  supported  the  Department  of  Defense  during  the  Virtual  Lifetime  Electronic  Record 
(VLER)  Phase  la  pilot. 

The  core  of  MIDHT  built  upon  the  successful  participation  in  the  VLER  pilot  and  provided  a  platform  in 
which  U.S.  veterans  in  rural  Pennsylvania  could  have  components  of  their  medical  record  seamlessly 
exchanged  electronically  between  Conemaugh  and  the  James  E.  Van  Zandt  VA  Medical  Center  (Altoona, 
PA).  At  the  conclusion  of  the  MIDHT  program,  Conemaugh  established  a  three  year  production 
exchange  of  live  patient  data  via  the  eHealth  Exchange,  including: 

•  data  domains  from  three  electronic  health  record  systems;  Allscripts  (outpatient), 
McKesson  (inpatient)  and  Sunquest  (laboratory) 

•  CCHIT  certification 

•  relevant  code  contributions  to  the  open-source  CONNECT  community 

•  an  ONC  supported  pilot  for  electronic  patient  consent 

Conemaugh  continually  applied  revisions  to  our  health  information  exchange  architecture  to  stay 
current  and  meet  partner  needs.  Brief  level  summarizations  are  included  within  this  report. 
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In  addition  to  the  technical  infrastructure  necessary  for  establishment  of  the  proposed  health 
information  exchange  (Subtasks  2. 1-2. 3),  MIDHIT  additionally  assessed  and  analyzed  NwHIN-related 
activities  (Subtask  2.4).  Within  that  scope,  the  following  research  objectives  were  investigated. 

Both  of  the  following  hypotheses  have  been  accepted: 

1.  The  use  of  the  health  information  exchange  (HIE)  via  the  NwHIN  -  CONNECT 
architecture  will  be  different  between  the  Conemaugh  Health  System  and  the 
Department  of  Veterans  Affairs. 

2.  Conemaugh  satisfaction  with  HIE  via  the  NwHIN  -  CONNECT  architecture  will  be  greater 
than  that  for  the  existing  (mostly  paper-based)  process  of  exchanging  health 
information  between  the  Conemaugh  Health  System  and  the  Department  of  Veterans 
Affairs. 

**  However  this  hypothesis  is  accepted  with  caution  due  to  small  sample  size  (n=14) 

Research  Objectives 

1.  Assess  and  analyze  production  data  center  performance  using  common  industry  metrics 

2.  Quantify  the  number  of  Conemaugh  patients  opted-in  and  successfully  correlated  with  VA  (i.e. 
relative  to  total  count  of  eligible  [common]  patients) 

3.  Quantify  provider  usage  of  the  NwHIN  Exchange 

4.  Evaluate  patient  satisfaction  of  the  NwHIN  Exchange 

5.  Evaluate  provider  satisfaction  of  the  NwHIN  Exchange  and  CONNECT  UniversalClientGUI 

Project  Activities,  Outcomes  and  Conclusions 

The  MIDHT  eHealth  exchange  "Gateway"  -  System  Overview 

As  per  The  Sequoia  Project  website  (http://sequoiaproiect.org/ehealth-exchange/).  the  eHealth 
Exchange  ("Exchange")  is  a  group  of  federal  agencies  and  non-federal  organizations  that  came  together 
under  a  common  mission  and  purpose  to  improve  patient  care,  streamline  disability  benefit  claims,  and 
improve  public  health  reporting  through  secure,  trusted,  and  interoperable  health  information 
exchange."  In  support  of  the  eHealth  Exchange  mission,  the  MIDHT  program  established  a  "Gateway" 
that  served  as  an  onramp  to  the  eHealth  Exchange  network. 

The  main  component  of  the  Conemaugh  "Gateway"  is  the  CONNECT  open-source  software,  which 
supports  the  following  sub  components: 

•  Core  Services  -  Facilitates  secure  messaging  services,  including  Patient  Discovery  (PD),  Query  for 
Documents  (QD),  and  Retrieve  Documents  (RD) 

•  Enterprise  Services  -  Also  known  as  the  "Adapter"  level,  these  services  bridge  the  Core  Services 
to  the  existing  Conemaugh  infrastructure 
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•  Universal  Client  Framework  -  Provides  user-facing  Graphical  User  Interfaces  (GUI)  that  are  used 
to  conduct  PD,  QD,  and  RD  queries  and  set/change  the  consent  directive  for  a  given  patient. 

The  CONNECT  open-source  software,  working  in  conjunction  with  an  open-source  registry/repository, 
an  open-source  access  control  platform  ("OpenSSO),  an  open-source  directory  service  ("OpenDS"),  a 
consumer-off-the-shelf  (COTS)  Enterprise  Master  Patient  Index  (EMPI),  and  COTS  EHRs,  enabled 
Conemaugh  to  share  structured  Continuity  of  Care  Documents  (CCD)  and  unstructured  C62  documents 
with  other  eHealth  Exchange  participants. 

Figure  54  depicts  Conemaugh's  eHealth  exchange  architecture. 


A  "Technical  Services  Agreement"  was  fully  executed  between  subcontractor  Northrop  Grumman 
Corporation  (NG)  and  CHS  on  October  26,  2010.  NG  had  staff  from  three  states  (Pennsylvania,  Utah  and 
Virginia)  supporting  the  contract  through  October  2013.  CGI  Federal,  Inc.  was  awarded  a  subcontract  on 
October  7,  2013  through  the  remainder  of  the  MIDHT  period  of  performance  (POP)  after  a  competitive 
selection  process  after  the  NG  contract  was  terminated.  Noteworthy  software  development 
deliverables  are  located  in 
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Figure  54.  Conemaugh's  eHealth  Exchange  Architecture. 
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Subtask  2.1  Deploy  a  limited  production,  NHIN  standards-based  HIE  focusing  on  the  bi-  directional 
exchange  of  electronic  medical  records  between  CHS  and  the  Military  Health  System.  CHS  information 
to  include  data  domains  residing  in  acute  care  and  ambulatory  settings. 

Conemaugh  and  subcontractors  have  worked  closely  to  design  and  implement  a  standards-based  health 
information  exchange  using  the  five  core  NwHIN  specifications,  which  include  patient  discovery, 
document  query,  document  retrieve,  messaging  and  authorization. 

NwHIN  Onboarding  Review 

In  orderto  participate  in  the  NwHIN,  participants  must  complete  various  phases  of  the  Onboarding 
process  before  exchanging  data.  Test  data  was  entered  by  MIS  staff  on  various  patients,  including 
demographics  (Initiate  MPI)  and  clinical  documents  (Allscripts).  Technical  issues  encountered  during 
various  testing  sessions  with  ONC  contractors  were  subsequently  resolved  during  each  phase  allowing  for 
conformance  and  interoperabilitytesting.  Conemaugh  was  conditionally  accepted  to  participate  in  the 
"Exchange"  on  November  20,  2011  (Appendix  I). 

Monthly  NwHIN  coordinating  committee  meetings  were  attended  throughout  the  process.  The  various 
phases  of  the  Onboarding  process  are  outlined  in  Table  38.  All  onboarding  activities  were  successfully 
completed  on  January  27,  2012.  Activities  included  operationalizing  the  Go-LIVE  endpoints  by  addition 
to  the  production  UDDI  and  installing  the  production  ONC  certificate  in  the  MIDHT  server. 


Onboarding  stages 

Stage  1:  Qualification 

^  After  self-qualification,  organization  submits 
completed  application  package  including  the 
signed  Data  Use  and  Reciprocal  Support 
Agreement  (DURSA) 

^  NwHIN  Implementation  Team  reviews 
application  package  and  works  with  the 
organization  as  needed  to  complete  the 
package 

^  NwHIN  Implementation  Team  reviews 
application  package  and  coordinates  an 
eligibility  review  with  the  NwHIN  Coordinating 
Committee  (NCC) 

^  NCC  reviews  application  package  to  ensure 
organization  meets  all  eligibility  requirements 

Stage  2:  Validation 

S NwHIN  Implementation  Team  sends  test 
certificate  and  validation  framework 
information  to  organization 

S Organization  configures  its  test  environment 
and  executes  conformance  and  interoperability 
testing 

S Organization  submits  evidence  of  successful 
tests  to  NwHIN  Implementation  Team 
^NwHIN  Implementation  Team  prepares 
validation  package  and  submits  to  NCC 
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Stage  3:  NCC  Review 

S NwHIN  Implementation  Team  coordinates  a 
review  with  NCC  to  evaluate  the  application  and 
validation  results 

■S NCC  evaluates  application  and  makes  a 
decision  on  whether  to  conditionally  approve 
the  organization  or  disapprove  the  organization 
and  request  remediation  (if  applicable) 

S NCC  notifies  the  NwHIN  Implementation  Team 
and  the  organization  of  membership  status 

Stage  4:  Activation 

^NwHIN  Implementation  Team  provides 
production  certificate  and  requests  production 
registry  information  from  the  organization 

S Organization  provides  production  registry 
information  to  NwHIN  Implementation  Team 

S NwHIN  Implementation  Team  configures  NHIN 
registry  with  organization's  information 
■S NCC  executes  the  DURSA  Joinder 

S NCC  notifies  organization  of  NHIN  membership 

S Organization  is  now  a  NwHIN  Participant  and 
ready  to  exchange  data  over  the  NwHIN 

Exchange 

Table  38.  Onboarding  Stages. 


Prior  to  deployment,  the  HIE  environment  was  updated  and  integrated  with  open-source  version 
CONNECT  3.1.  Two  web  services  required  updates  to  match  the  WSDLs  in  CONNECT  v3.1.  Other 
integration  issues  were  resolved  by:  1)  eliminating  "Dispatch"  errors  through  the  condensation  of 
endpoints  in  the  Document  Manager  web  service,  and  2)  redirecting  the  hibernate  code  to  local 
configuration  files.  Other  work  included:  patient  discovery  with  Initiate  Master  Patient  Index, 
restructuring  of  xml  consistent  with  HITSP  C32  specification,  integration  of  emergency  room  discharge 
summaries  in  HITSP  C62  format  and,  completing  the  NWHIN  conformance/interoperability  testing  with 
the  ONC.  Please  refer  to  Appendix  J  and  Appendix  K  for  C32  and  C62  examples  respectively. 

The  following  three  components  were  critical  to  develop  and  test  before  individual  health  data  could  be 
exchanged  (namely.  Patient  Discovery,  Patient  Consent,  and  Patient  Matching). 

Patient  Discovery 

The  Patient  Discovery  service  was  based  off  of  the  eHealth  Exchange  Patient  Discovery  specifications, 
which  defines  how  patient  matching  occurs  across  the  network.  The  Patient  Discovery  service  provided 
a  mechanism  in  which  the  MIDHT  "Gateway"  could  query  another  eHealth  Exchange  participant  to 
determine  if  the  same  patient  existed  at  the  participant's  organization.  Likewise,  a  remote  eHealth 
Exchange  participant's  organization  could  query  the  MIDHT  "Gateway"  to  see  if  a  matching  patient 
existed. 
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The  Patient  Discovery  process  was  initiated  when  demographics  (e.g.  First  Name,  Last  Name,  Date  of 
Birth  [DOB],  Gender,  and  Social  Security  Number  [SSN])  associated  with  a  patient  were  extracted  from 
the  requesting  organization's  EMPI,  and  then  broadcasted  to  other  Exchange  participants.  Only,  and  if 
only,  a  single  matching  patient  was  found  at  the  responding  organization,  the  responding  organization 
returned  a  response  containing  a  patient  identifier  and  the  patient's  set  of  demographics.  Upon  receipt 
of  the  response,  the  requesting  organization  verified  the  demographics  against  their  local  EMPI  to 
ensure  that  both  agree  on  the  patient's  identity.  If  an  agreement  was  reached  between  two 
organizations  that  a  single  patient  match  had  been  achieved,  then  the  patient  identifier  from  the 
requesting  gateway  was  correlated  with  the  patient  identifier  from  the  responding  gateway  and  saved  in 
a  patient  correlation  repository.  Clinical  users  were  then  able  to  view  correlations  for  a  given  patient  via 
the  CONNECT  Universal  Client  GUI  web  portal  (Figure  55). 
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Figure  55.  Patient  Correlations  Displayed  in  CONNECT  Universal  Client  GUI. 

In  the  case  of  the  responding  gateway  returning  multiple  patient  matches,  the  requestor  received  a 
response  that  contained  a  no  match  result  -  which  was  their  original  web  service  request  contained 
within  the  response. 

Patient  Consent 

The  MIDFIT  patient  consent  model  was  that  of  "opt-out"  by  default,  where  CHS  was  not  authorized  to 
share  a  patient's  information  until  a  signed  paper  consent  document  was  obtained  from  the  patient. 
Before  patient  information  was  shared  between  organizations,  both  CHS  and  VA  had  to  receive  consent 
from  a  patient  to  release  his/her  medical  information.  Once  consent  was  obtained,  the  patient  was 
"opted-in"  by  Conemaugh  staff  using  the  Consumer  Preferences  Profile  (CPP)  GUI  (Figure  56). 
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Consumer  Preferences  Profile 


Search  Patient 
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CPP  Client  User's  Manual 
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C  NHIN  Opt-Out 


Save  Preferences 

Figure  56.  CPP  GUI 

Patient  Matching 

Patient  matching  accuracy  was  of  critical  importance  during  the  Patient  Discovery  process.  For  the 
MIDHT  project,  an  identity  resolution  strategy  that  stressed  probabilistic  matching  for  verifying  the 
identity  of  a  patient  was  implemented  to  identify  a  single  patient  from  mutable  and  immutable 
identifiers.  To  achieve  the  best  possible  match,  a  matching  strategy  that  provided  identity  integrity 
addressed  the  following  key  areas  that  could  impact  the  matching  of  records: 

•  Quality  of  Data  -  A  good  matching  solution  must  utilize  strong  identifiers,  must  address 
variations  in  syntax,  differences  in  semantics,  and  must  account  for  missing  and  incomplete  data 
values. 

•  Quality  of  Matching  Criteria  -  The  matching  criteria  associated  with  the  deployed  algorithm  must 
be  set  high  enough  to  diminish  the  presence  of  false  positive  results  (information  for  multiple 
patients  appear  to  be  a  match  for  a  given  set  of  demographics)  as  well  as  a  small  number  of  false 
negatives  (multiple  records  for  the  same  patient  are  returned),  but  not  so  high  that  no  results  are 
returned. 

•  Quality  of  the  Matching  Algorithm  -  The  matching  algorithm  shall  support  the  best  matching 
accuracy. 

Because  the  eHealth  Exchange  does  not  offer  the  benefit  of  a  National  Patient  Identifier  (NPI), 
unattended  probabilistic  matching  solutions  offer  the  best  chance  at  finding  a  patient  match.  In  the 
case  of  the  MIDHT  "Gateway",  probabilistic  matching  was  tuned  using  a  min  score  criteria  to  meet 
customer  defined  matching  requirements. 

Since  the  Patient  Discovery  process  was  so  heavily  dependent  on  making  matches  via  probabilistic 
matching  algorithms,  it  was  imperative  that  min  score  criteria  be  adjusted  to  account  for  missing  or 
incomplete  data.  During  implementation  of  the  MIDHT  "Gateway",  the  EMPI  min  score  needed  to  be 
"dialed-in"  to  a  setting  that  could  best  support  patient  demographics  containing  varying  levels  of 
information  robustness  and  completeness.  Inclusion  of  the  patient's  Social  Security  Number  (SSN)  was  a 
great  benefit  to  our  correlation  efforts.  Not  all  VA  partners  included  this  demographic  field. 
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Query  for  Documents 


The  Query  for  Documents  service  enables  an  eHealth  Exchange  participant  to  request  a  list  of  clinical 
documents  from  other  Exchange  participants.  However,  the  Query  for  Documents  service  is  dependent 
on  the  Patient  Discovery  service,  and  requires  a  patient  to  be  correlated  between  organizations  before 
documents  can  be  exchange  between  those  organizations.  The  Query  for  Documents  process  begins 
when  an  eHealth  Exchange  participant  sends  a  Patient  Identifier  (PID),  obtained  during  Patient 
Discovery,  to  a  remote  participant  to  identify  the  patient  for  which  they  would  like  a  list  of  documents. 
The  remote  organization  then  returns  a  list  of  documents  pertaining  to  the  requested  patient. 

As  per  the  2011  eHealth  Exchange  specifications,  the  documents  in  the  returned  list  are  classified  as 
"Stable"  or  "On-Demand"  documents.  "Stable"  documents  are  documents  that  contain  metadata 
describing  the  document  when  it  was  created  and  made  available  for  retrieval.  On  the  other  hand,  On- 
Demand  documents  contain  metadata  describing  a  document  that  is  to  be  created  when  the  requestor 
attempts  to  retrieve  it.  When  the  MIDHT  "Gateway"  receives  a  Query  for  Documents  request,  a  list  of 
documents  is  returned  to  the  requesting  organization  -  but  only  after  the  document  has  been 
dynamically  created  from  information  that  originates  from  CHS's  Electronic  Health  Record  (EHR) 
systems.  Though,  the  MIDHT  Query  for  Documents  response  is  similar  to  what  is  described  in  the  On- 
Demand  specification,  documents  returned  by  the  MIDHT  "Gateway"  are  considered  to  be  "Stable" 
documents  because  the  document  and  its  metadata  are  created  during  the  Document  Query  process 
and  made  available  for  retrieval. 

Dynamic  Document  Generation 

When  a  Query  for  Documents  request  is  received  by  the  Conemaugh  gateway,  a  C32  and/or  C62  clinical 
document  is  dynamically  created  using  a  Dynamic  Document  Generation  plug-in  for  the  CONNECT 
architecture,  and  the  metadata  about  each  created  document  is  returned  to  the  requestor.  The  Dynamic 
Document  Generation  architecture,  which  is  comprised  of  a  Document  Assembly  Web  Services 
Interface,  a  Templates  component,  a  Document  Assembly  component,  a  Document  Repository 
component  and  a  Common  Data  Layer  Web  Services  Interface,  is  responsible  for  transforming  raw  data 
from  Conemaugh's  EHR  systems  into  structured  and  unstructured  clinical  documents. 

The  process  begins  when  CONNECT  passes  an  incoming  Document  Query  request  to  the  Document 
Assembly  Web  Services  Interface,  alerting  the  Dynamic  Document  Generation  architecture  that  a  clinical 
document  has  been  requested.  The  Document  Assembly  component  then  receives  the  request  from  the 
Document  Assembly  Web  Service  Interface  and  begins  the  process  of  building  a  Clinical  Document 
Architecture  (CDA)  document.  The  CDA  document  construction  process  begins  when  the  Document 
Assembly  Manager  component  calls  the  Templates  Manager  to  retrieve  template  information  from  a 
database  table.  The  database  table  contains  information  regarding  which  modules  and  sections 
comprise  a  particular  type  of  CDA  document  and  which  modules  and  sections  have  been  enabled  by  the 
gateway  administrator.  The  information  retrieved  from  the  templates  database  table  is  then  used  to 
make  calls  to  the  Common  Data  Layer  Web  Services  Interface,  which  returns  information  from  the  EHR 
systems.  The  data  is  aggregated  into  a  clinical  document,  which  is  saved  in  the  document  repository, 
along  with  its  associated  metadata,  via  the  Document  Repository  component.  The  document 
information  is  then  extracted  from  the  repository  and  packed  into  a  Document  Query  response,  which  is 
returned  to  the  requestor. 
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One  of  the  keys  to  responding  to  Document  Query  requests  with  dynamically  created  clinical  documents 
is  to  ensure  that  the  EHR  systems,  from  which  you  extract  the  data,  have  mechanisms  in  place  for 
exporting  data.  The  ridged  architecture  of  the  Commercially  Off-The-Shelf  (COTS)  EHRs  at  Conemaugh 
had  to  be  augmented  with  a  custom  solution  to  ensure  that  the  required  clinical  data  could  be  extracted 
from  the  EHRs  and  packaged  in  a  format  suitable  for  transmission  across  the  eHealth  Exchange. 

Another  key  to  dynamically  creating  CDA  documents  is  ensuring  that  the  mapping  of  data  from  the  EHR 
systems  to  the  CDA  documents  is  accurate  and  complete.  Often  times,  the  format  and/or  content  of  the 
EHR  data  are  not  conducive  to  meeting  the  CDA  specification.  To  address  this  issue,  the  MIDHT 
"Gateway"  deploys  a  series  of  custom  "mappers"  that  transform  the  content  and  structure  of  certain 
pieces  of  data  as  the  CDA  is  being  created. 

Document  Retrieve 

The  Document  Retrieve  services  provide  a  mechanism  in  which  an  eHealth  Exchange  participant  is  able 
to  retrieve  one  or  multiple  clinical  documents  from  a  remote  participant.  The  services  are  utilized  after  a 
unique  document  identifier  is  received  in  a  successful  Document  Query  response.  This  unique  identifier 
is  used  in  the  Document  Retrieve  request  to  identify  which  document(s)  should  be  returned  to  the 
requesting  participant.  When  a  Document  Retrieve  request  is  received  by  the  MIDHT  architecture,  the 
Document  Repository  component  is  responsible  for  querying  and  retrieving  the  document(s)  from  the 
document  repository. 

One  of  the  challenges  of  the  retrieving  clinical  documents  from  a  remote  eHeatlh  Exchange  participant  is 
making  sure  the  documents  are  viewable  when  they  are  received.  Since  clinical  information  in  a  CDA 
document  is  represented  as  both  summary  information  and  information  in  the  body  of  the  CDA 
document,  an  organization  must  decide  how  they  will  parse  and  display  the  information.  As  Conemaugh 
learned  when  they  first  began  exchanging  data  with  the  VA,  one  style  sheet  is  not  conducive  to 
displaying  information  from  disparate  organizations.  Instead  of  using  the  style  sheet  that  Conemaugh 
had  been  using  to  view  their  structured  data,  a  new  style  sheet  had  to  be  developed  in  order  to  view  the 
content  of  clinical  documents  that  are  returned  by  the  VA. 

2.1.1  VLER  Exchange  Activities  -  Conemaugh  and  James  E.  Van  Zandt  VA  Medical  Center 

The  VLER  program  was  discussed  with  various  healthcare  groups  within  Conemaugh  and  the  VA. 

Meetings  focused  on  various  topics,  including  the  eHealth  Exchange,  clinical  data  availability,  system 
usage,  and  qualitative  surveys.  First  sample  documents  from  the  VA  were  received  in  November  7,  2011. 
VA  C62  class  codes  were  received  approximately  one  month  later  and  added  as  an  option  in  the  doc 
query  request  to  the  VA.  Testing  began  in  January  2012  with  the  first  production  document  transaction 
occurring  in  October  2012. 

An  initial  13  patients,  who  were  "opted  in"  at  the  VA  Johnstown  Community  Based  Outpatient  Clinic  (a 
CBOC  of  the  James  E.  Van  Zandt  VA  Medical  Center)  and  successfully  correlated  via  patient  discovery, 
served  as  the  data  set  for  validation  testing  (against  production  data).  Additional  meetings  between 
stakeholders  were  then  used  to  also  discuss  the  availability  of  Conemaugh  C32  data  and  other  data 
quality  issues. 

On  September  11,  2012  Conemaugh  provided  and  extensive  patient  list  to  the  VA  that  included  116,616 
patients  with  C32  and  C62  data  available  from  the  past  two  years.  The  VA  performed  a  match  against  the 
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patient  population  at  the  James  E  Van  James  E.  Van  Zandt  VA  Medical  Center  -  Altoona,  PA,  creating  a 
"shared"  patient  list  that  was  then  used  for  a  mass  US  postal  mailing  of  consent  forms.  For  patients  who 
did  not  have  a  primary  care  physician  on  file.  Allscripts  EHR  was  used  to  uncover  the  information. 

Before  distribution,  the  patient  file  was  reviewed  for  deceased  patients  and  address  changes.  Patient 
lists  were  then  generated  for  each  provider  practice  that  was  first  piloting  the  program  and  shared  for 
review. 

In  October  2012,  over  355  veterans  were  mailed  the  joint  patient  information  letter  (Appendix  L),  two 
consent  forms  (Appendix  M  and  Appendix  N),  and  marketing  materials  (Appendix  0)  regarding  the 
exchange.  MIDHT  staff  'opted-in'  specific  patients  through  the  CONNECT  CPP.  The  VA  forms  were 
provided  on  a  weekly  as  needed  basis.  Patient  correlations  were  then  checked  frequently  to  determine 
the  attars  of  the  linkage.  Once  a  correlation  was  made  between  Conemaugh  and  the  VA,  MIDHT  staff 
added  chart  alerts  within  the  Allscripts  electronic  health  record  to  notify  providers  (Figure  57). 


Figure  57.  Adscripts  Chart  Alert. 

A  second  patient  list  consisting  of  patients  that  had  been  referred  from  the  VA  to  Conemaugh  in  the  past 
two  years  (2012-2014)  (approximately  300)  also  received  program  information.  It  was  decided  that  the 
Altoona  VAMC  would  manage  this  manage  mailing  moving  forward.  Discussions  with  the  Department  of 
Veterans  Affairs  (VA)  regarding  future  patient  mailings  had  been  ongoing  throughout  the  period  of 
performance. 

Conemaugh  utilized  internal  marketing  resources  to  promote  the  VA  partnership  and  VLER  project. 
Content  was  distributed  via  the  following  means:  an  employee  newsletter,  physician  newsletter, 
individual  provider  office  meeting,  and  press  release  (with  advance  review  byTATRC)  (Appendix  P). 

Working  with  administration,  ambulatory  users  primary  from  family  medicine  were  informed  of  the 
project  and  training  then  commenced.  Individual  system  accounts  were  created  to  allow  end  users  to 
request  data. 

From  May  2012  through  September  2015, 1,150  patients  returned  authorizations  with  1,136  opting-in 
to  the  service  (NwHIN  exchange)  and  1,135  correlations  (Figure  58).  The  exciting  milestone  of  1,000 
correlations  was  achieved  on  June  9,  2014.  Of  those  correlated: 

•  99.7%  (1,103/1,136)  were  male 

•  For  those  opting  in  and  successfully  correlated,  the  average  age  was  71  years 

•  99%  (1,136/1,150)  success  rate  for  patient  discovery  messages 
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Figure  58.  eHealth  Exchange  Return  Authorizations  and  Correlations 

The  return  rate  of  authorizations  was  directly  correlated  to  the  joint  mass  mailings  spearheaded  by 
Conemaugh.  The  difference  in  the  number  of  veterans  that  responded  and  are  correlated  is  not  due  to 
patient  discovery  failures  but  rather  issues  with  VLER  authorizations  (e.g.  not  signed/dated,  not 
completed)  and  the  time  delay  for  both  organizations  to  "Opt-ln"  patients  into  their  system.  Again, 
specific  patient  information  could  not  be  exchanged  through  the  eHealth  Exchange  until  permission  was 
granted  from  the  Veteran. 

The  incorporation  of  lab  results  from  the  Sunquest  lab  system  into  the  CONNECT  environment  was 
requested  by  VA  users  and  was  a  major  project  milestone.  Conemaugh  first  began  work  with  Sunquest 
on  the  HL7  interface  for  laboratory  results  during  February  2014.  Live  testing  of  the  Sunquest  interface 
with  live  data  posed  problematic,  and  as  a  result  restrictions  were  placed  on  microbiology  and  blood 
bank  results.  John  Hargreaves  worked  with  the  CONEMAUGH  lab  to  provide  documentation  to  meet  the 
College  of  American  Pathology  (CAP)  interface  requirements. 

The  first  successful  lab  query  response  was  completed  on  March  13,  2014.  A  handful  of  issues  including 
integration  into  the  CONNECT  environment  and  C32  document  were  resolved  during  the  summer  before. 
Testing  with  the  VA  was  regarding  the  laboratory  results  were  completed  on  August  18,  2014. 

Throughout  the  POP,  Conemaugh  upgraded  Allscripts  software  versions.  MIDHT  integration  and 
functionality  was  changed  as  needed,  tested  and  put  back  into  production  efficiently. 

2.1.2  VLER  Exchange  Usage 

Successful  document  retrieve  transactions  for  treatment  purposes  through  the  eHealth  Exchange 
increased  overtime  but  did  vary  (Figure  59).  Figure  60  is  an  overall  representation  by  document  type  with 
Figure  61  displaying  exchanged  documents  by  direction  of  receipt.  Data  is  continuous  on  a  monthly 
basis. 
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eHealth  Exchange  Transactions 


Month  -  Year 


Figure  59.  eHealth  Exchange  Transactions  during  POP. 


Month  -  Year 


CCD_0  =  Continuity  of  Care  Document  Outbound 
RAD-0  =  Radiology  Report  Outbound _ 

DS-0  =  Discharge  Summary  Outbound 
ER_0  =  Emergency  Room  Note  Outbound 


Figure  60.  eHealth  Exchange  Transactions  by  Document  Type. 
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Month  -  Year 


CCD_I  =  Continuity  of  Care  Document  Inbound 
RPT_I  =  C62  Reports  Inbound 

Figure  61.  Inbound  vs.  Outbound.  Continuity  of  Care  Documents. 

Conemaugh  providers  prefer  retrieving  VA  lab  results  and  medication  history  via  the  continuity  of  care 
document  (CCD).  Retrieval  of  C62  reports  from  the  VA  was  minimal  and  highly  influenced  by  one 
user/day.  Likewise,  VA  providers  have  preferred  to  retrieve  the  CCD  from  Conemaugh  and  also 
radiology  reports.  Retrieval  of  emergency  room  notes  and  discharge  summaries  have  also  been 
minimal.  Interestingly,  the  initiation  of  document  retrievals  is  nearly  a  50/50  split. 

Allscripts  Clinical  Analytics  software  was  made  available  to  Conemaugh  in  May  2014.  The  purchase 
agreement  was  executed  with  the  vendor  six  months  prior  (November  1,  2013).  Staff  training  was 
conducted  via  webinars  and  workshops.  Clinical  data  on  veterans  was  extracted  according  to  2014 
American  Diabetes  Association  (ADA)  standards.  Data  was  consolidated  for  presentation  to  system 
users.  The  MIDHT  team  worked  closely  with  the  CPG  Director  of  Primary  Care  on  review  and 
distribution  of  the  data  to  physician  practices.  The  initial  dataset  was  distributed  on  September  3,  2014, 
with  intentions  to  increase  document  queries  by  Conemaugh  users  by  analyzing  existing  clinical  data  in 
the  electronic  record  and  comparing  it  to  national  standards  in  order  to  identify  gaps  in  care. 
Unfortunately  document  queries  by  Conemaugh  for  diabetic  patients  was  less  than  five  and  did  not 
produce  a  realizable  value. 

2.1.3  Direct  Exchange  Push  vs.  Query-Based  Exchange  Comparison 

The  following  analysis  compares  the  direct  exchange  push  functionality  purchased  from  McKesson 
currently  in  a  limited  production  state  and  the  query-based  approached  currently  in  production  via  the 
eHealth  Exchange  and  Virtual  Lifetime  Electronic  Record  (VLER)  program. 
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Direct  Exchange  Push  InboumJ 


When  a  patient's  clinical  document  architecture  (CDA)  is  sent  to  our  health  system  from  an  external 
source  and  attached  to  the  patient  record,  the  information  is  visible  to  CHS  providers  under  the 
Transcription  tab  in  Care  Portal  as  an  Exchange  Message-Inbound  CDA  (Figure  62).  See  Appendix  Q  for 
specific  inbound  documentation  instructions. 
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-  I  g  I  > 


my  portal  |  [suspend]  [  help |  [logout  [ 


Def  Census  Search  Results  Trans  Cardio  Sum  Viewer  DocView  eMAR  Orders  Med  Record  Home  Facesheet  CPOE  CPG  PMH  Medical  Reconciliation 


|  <Select> 


“3 


Patient:  I  DORSEY,  DEBBIE  DORKIS;  000911876  ;G6-5555-l  |  |  Encounter:  1 12/05/13; ;  DR  NOT  IN  FILE  (a) 


CD  (HD  CM  Demographics  JSR  Portlet 

last  refresh:  19: 15  (fi) (© (2>) 

Pat  Name 

Sex  Birth  Date 

Age  Location 

MR 

Number 

Enc  ID/Acct 
Number 

Allergies 

.  ,  _  .  Disc 

Adm  Date  . 

Date 

Adm 

Diagnosis 

Attending 

DORSEY, 

DEBBIE 

DORKIS 

F  01/01/60 

00:00 

54  G6-5555- 

Y  1 

000911876 

1701610 

morphine,  Sulfa  (Sulfonamide  Antibiotics),  Shellfish 
Containing  Products,  latex,  egg,  iodine,  Grapefruit, 
pollen  extracts 

12/05/13 

15:58 

DR  NOT  IN 
FILE 

CD  (jdTT)  cm  Transcription 


last  refresh:  19:13  (©©CD 


Report  for  DORSEY,  DEBBIE  DORKIS  (MRN:  000911876) 
Results  types  |  ALL  Transcriptions  Result  test  classes  |  Edit  Preference  for  ALL  Transcriptions 

F  Start  On:  |01/24/2014  Go  Back:  f30  Days  T  New  reports  T  Today’s  reports  T  Sort  reports  by  test  class 
Show  Selected  Reports _ |  Show  All  Reports  |  item  Search:  f 


Clear 


Refresh 


Viewed  Date  Links  Description  Message  Subject  From  Status  Hst 

r  01/23/14  14:10  Exchange  Message  -  Inbound  CDA  Referral  for  New  Patient  diane.bengtson0748@direct.healthvault.com  Amended  % 

r  01/23/14  13:42  Exchange  Message  -  Inbound  CDA  Patient  information  for  Debbie  Dorkis  Dorsey  diane.bengtson0748@direct.healthvault.com  Amended  % 

Show  Selected  Reports |  Show  All  Reports 


(Back  to  Top) 


MCKESSON  I  HORIZON* 


Figure  62.  Inbound  CDA  Message  Exchange. 


Hard  copy  printing  is  enabled,  as  well  as  the  ability  to  send  important  information  to  the  patient's 
electronic  McKesson  chart  for  review  and  reconciliation  by  the  nursing  staff.  Information  may  include: 
problems,  allergies,  medications,  etc. 


Direct  Exchange  Push  Outbouncj 


Conversely,  CHS  providers  can  also  push  CDA  documents  to  external  providers  and  patient  portals. 
The  specific  workflow  is  outlined  in  Appendix  R. 


Queryzbased  Exchange 

The  VLER  Exchange  architecture  uses  secure  robust  eHealth  Exchange  protocols  and  standards  to 
support  query-based  exchange  use  cases  nationwide.  In  our  case,  interoperable  data  exchange  supports 
the  treatment  of  veterans  in  Southwestern  Pennsylvania  between  a  private  healthcare  organization  (i.e. 
Conemaugh)  and  the  Department  of  Veterans  Affairs.  After  receipt  of  the  patient's  authorization  form, 
opt-in  and  successful  correlation  between  organizations,  Conemaugh  providers  can  query  the  VA  system 
via  the  CONNECT  Universal  Client  GUI  (Figure  63)  for  a  C-CDA  (Figure  64).  Unstructured  reports  have 
recently  been  integrated  into  this  document  format  as  well. 
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Figure  63.  C-CDA  returned  within  CONNECT  Universal  Client  GUI. 

Click  on  the  hyperlink  to  view  the  document  contents.  Print  and  scan  as  needed. 
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Figure  64.  Sample  C-CDA  document. 

VA  providers  follow  a  similar  process  using  VistAWeb  (Figure  65). 
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Figure  65.  VistAWeb  from  VA. 

The  following  table  (Table  39)  compares  the  pros  and  cons  of  each  approach.  Ideally,  the  approach 
chosen  will  depend  on  the  specific  use  case  and  the  available  resources  to  support  the  infrastructure. 


Direct 

Exchange 

Pros 

Cons 

C-CDA  structured  in  human  readable 

format 

No  easy  lookup  to  determine  Direct  Exchange 
address,  contact  provider  for  information 

No  MPI  matching  needed 

Limited  push  availability  in  rural  areas  (IT 
support  and  knowledge  challenging) 

Authorization  covered  under  standard 
hospital  releases 

Manual  process  that  requires  a  staff  resource 
24  x  7  to  push  documents  to  parties 

Technical  uplift  much  lighter 

Multiple  connection  failure  points  in 
reception  (e.g.  manual  pushes  vs.  interface) 

Date  ranges  configurable 

Set  automatic  triggers  when  updates 
made  in  the  record  for  push  function 

Query- 

based 

Exchange 

Clinical  information  provided  quickly  to 
treating  provider  when  needed 

Technical  uplift  much  greater 

Various  document  types  available  (e.g. 
C-CDA,  C32,  C62) 

Requires  skilled  programmers  and 
knowledgeable  contractors 

Consent  improves  patient  awareness  of 

MPI  matching  required  between  systems 
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initiative 

(correlation) 

100+  organizations  in  the  network, 

Patient  authorization  time-consuming  and 

including  government  agencies 

laborious 

Avoids  building  custom  interfaces  with 

Identification  of  correlated  patients  to  end 

trading  partners 

users  can  be  tricky 

Mature  trust  policies  that  are  widely 
adopted  (e.g.  DURSA) 

Table  39.  Direct  based  vs.  Query  based  Exchanges. 


Subtask  2.2  Provide  technical  and  documentation  assistance  on  DoD-managed  Virtual  Lifetime 
Electronic  Record  (VLER)  efforts. 

******l\|0  specific  tasking  was  identified  by  TATRC  for  this  contract****** 

Subtask  2.3  Investigate  productizing  a  Patient  Consent  module  using  established  standards, 
such  as  TP20/XACML. 

Conemaugh  and  NGC  agreed  to  participate  in  the  Jericho/UT-Austin  DS4P  Pilot  in  July  2012  by  serving  as 
a  test  CONNECT  gateway  on  a  simulated  eHealth  Exchange.  Pilot  work  was  not  initiated  until  April  2013 
due  to  contracts  and  funding  issues.  Information  about  the  pilot  can  be  found  at: 
http://wiki.siframework.org/Data+Segmentation+for+Privacv+Homepage 

The  pilot  explored  12  electronic  exchange  scenarios  and  eight  types  of  data  transactions.  Pilot 
participants  performed  various  roles  to  test  these  scenarios.  Jericho  Systems  served  as  primary 
custodian  of  the  patient's  record  and  housed  the  Patient  Consent  Directive  (PCD)  repository.  The 
University  of  Texas  at  Austin  HIT  Program  simulated  the  role  of  a  research  university  that  requested  the 
patient's  record  and  subsequently  acted  as  the  secondary  custodian  of  the  information.  Conemaugh 
played  the  role  of  a  marketing  network  that  became  the  second  requestor. 

In  support  of  the  DS4P  J-UT  pilot,  a  CentOS  Virtual  Machine  (VM)  image  was  successfully  retrieved  from 
Jericho  and  installed  on  a  MIDHT  test  sever  in  August  2013.  Once  installed,  the  GlassFish  server 
contained  within  the  VM  was  configured  to  use  ports  443  and  80  instead  of  ports  8181  and  8080. 

NCG  additionally  helped  the  Jericho  technical  team  establish  a  GoTo  Meeting  session  that  was  used  to 
support  the  Jericho/UT-Austin  Pilot  use-case  demonstration  (September  19,  2013).  NGC  continued  to 
provide  support  to  Jericho  until  contract  expiration. 

Subtask  2.4  Assess  and  analyze  NwHIN-related  activities,  to  include  data  center  performance 
metrics,  physician  evaluation  and  usage  of  the  NHIN  Portal,  and  resulting  benefits  of  HIE  with 
federal  participants. 

2.4.1  Patient  Correlations  (Patient  Discovery) 

As  previously  discussed  in  Subtask  2.1.1  (VLER  Exchange  Activities  -  Conemaugh  and  James  E.  Van  Zandt 
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VA  Medical  Center),  patient  correlations  were  seemingly  dependent  on  the  patient  response  to  direct 
postal  mailings  of  VLER  recruitment  packets.  Of  important  note,  failed  patient  correlations  were  often 
less  than  5%;  again  likely  resulting  from  inclusion  of  the  patient  Social  Security  Number  during  the 
matching  process.  This  rate  is  lower  than  that  noted  through  anecdotal  discussions  with  other  sites. 
Statistics  are  highlighted  below  in  Table  40  and  Table  41. 


Statistics 


Monthly_Count_ 

correl 

Monthly_Count_ 

Returned_Auths 

Count_Opted_IN 

_by_month 

Count_Discovered 

_by_month 

N 

Valid 

41 

41 

41 

41 

Missing 

0 

0 

0 

0 

Mean 

27.68 

28.05 

27.71 

27.68 

Median 

12.00 

12.00 

11.00 

12.00 

Mode 

0 

0 

0 

0 

Sum 

1135 

1150 

1136 

1135 

Table  40.  Patient  Correlation  Statistics. 


Sex_ 

male 

Sex_ 

female 

Age_AVG_ 

Opt_IN_yrs 

AVG_days_bw_ 

PT_disc_OPT_in 

N 

Valid 

41 

41 

36 

36 

Missing 

0 

0 

5 

5 

Mean 

26.9 

0 

.80 

69.47 

15.47 

Median 

10.0 

0 

.00 

70.23 

11.18 

Mode 

0 

0 

55.53 

.0000 

Sum 

1103 

33 

2501.01 

556.87 

Table  41.  Patient  Correlation  Demographics. 


2.4.2  Patient  Survey 

A  total  of  119  Veterans  submitted  responses  to  the  survey  and  97%  consented  to  share  their  health 
information  with  NwHIN  participants.  When  asked  if  they  felt  their  health  information  was  secure  and 
private,  72%  answered  YES  and  28%  were  NOT  SURE.  Forty-four  percent  of  the  patients  said  that  their 
providers  talked  about  the  new  service  (NwHIN  exchange)  during  their  appointment.  This  implies  that 
the  patient  perception  of  security  and  privacy  was  independent  of  provider  discussion  of  health 
information  exchange. 

Respondent  Demographics: 

•  95%  (112/118)  were  male 

•  89%  (106/119)  were  61  years  or  older,  with  the  remaining  age  18  -  60 

Veterans  responding  to  the  survey  have  had  a  positive  experience  with  health  information  exchange 
between  Altoona  VA  and  Conemaugh.  Over  60%  of  patients  believe  that  coordination  of  care  across 
providers  has  improved.  Similar  percentages  were  obtained  regarding  reducing  the  need  for  Veterans 
to  hand  carry  their  paper  records  to  providers  and  less  duplicate  testing.  Of  note  for  those  that  chose 
these  three  benefits,  at  least  72%  of  the  respondents  also  felt  their  health  information  was  secure  and 
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private.  More  than  half  of  survey  respondents  believe  that  decision  making  and  quality  of  care  has  also 
improved  (Figure  66). 


Q6  Please  select  which  benefits  you  believe 
were  achieved  as  a  result  of  HIE  between 
Conemaugh  and  Altoona  VA. 

Answered:  105  Skipped:  14 


Better 

Improved 

Improved 

Less 

Reduced 

1  am  Not 

Coordination 

Patient 

Decision 

Duplicate 

the  Need  to 

Sure 

of  Care 

Safety 

Making/ 

Testing 

Hand  Carry 

Quality  of  Care  Records 


Figure  66.  Patient  Perceived  Benefits  of  Health  Information  Exchange  (HIE). 


2.4.3  Provider  Survey 

Twenty-three  providers,  96%  Family  Medicine/Primary  Care  and  4%  Internal  Medicine,  submitted 
responses  to  the  survey,  with  91%  being  employed  by  the  Conemaugh  Health  System  and  9%  employed 
by  an  independent  practice.  The  Altoona  VA  did  not  provide  permission  to  survey  their  users,  which  was 
unfortunate.  Respondents  self-identified  as  a  member  of  one  of  the  following  categories,  physician 
(17%),  clinical  staff  (43%),  clerical  staff  (30%)  and  administration  (9%).  The  distribution  of  the 
respondents'  years  of  experience  follows  (Figure  67): 
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024  Years  of  experience: 

Answered:  23  Skipped:  0 


under  2 


2  to  5 


6  to  10 


11  to  15 


16  to  20 


21  to  25 


over  25 


0%  1 0%  20% 

Figure  67.  Provider  Years  of  Experience. 


30%  40%  50%  60%  70%  80%  90%  100% 


Most  of  those  surveyed,  61%  (14/23),  indicated  that  they  accessed  PHI  through  the  NwHIN/VLER 
Exchange.  For  those  who  did  not  (n=9),  44%  responded  that  they  'did  not  treat  Veterans',  33%  as  'not 
having  time',  and  22%  believed  'additional  training  was  needed'  (Q04)  -  consistent  with  79%  of 
respondents  having  found  the  training/user  manual  helpful  (Q22).  Nearly  all  those  who  indicated  using 
the  service  (93%)  believed  they  understood  the  steps  needed  to  retrieve  a  clinical  document  (Q20). 
Seventy-nine  percent  accessed  information  from  their  personal  office  while  the  remaining  21%  did  so 
from  a  clinic/unit/  or  patient  floor  (Q05).  The  types  of  documents  accessed  were  as  follows  (Qll)(Figure 
68): 
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Q11  Please  indicate  which  documents  you  have  accessed 


through  the  NwHIN/VLER  Exchange?  (Please  select  all  that  apply) 

Answered:  14  Skipped:  9 


Continuity  of 
Care  Doc.  (includes  lab  results) 


Discharge 

Summary 


ER  Reports 


History  & 
Physical  Note 

Consultation 

Note 


Radiology 

Report 


Procedure  Note 


Surgery  Report 


Pathology 
Report 

0%  10%  20%  30%  40%  50%  60%  70%  S0%  90%  100% 


■ 


Figure  68.  Provider  Exchange  Document  Type. 

When  asked  about  the  usability  of  the  service,  the  provider  participants  replied  as  follows 
(Figure  69): 

•  64%  agreed  that  the  GUI  is  clean  and  easy  to  navigate  (Q19) 

•  86%  agreed  that  clinical  data  is  presented  in  a  readable  format  (Q21) 

•  50%  perceived  the  processing  time  of  a  document  query  to  be  between  2-5 
minutes  (Q14) 

•  36%  perceived  the  processing  time  of  a  document  query  to  be  between  %  and  2 
minutes 
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Q6  Please  indicate  the  extent  to  which  you  agree  or 
disagree  with  the  following  statements: 
(Please  select  one  response  per  row) 

Answered:  14  Skipped:  9 


100% 

80% 


NwHIN/VLER  has  I  access  information  NwHIN/VLER  has 

reduced  the  time  I  must  from  the  VA  more  facilitated  consultation 

wait  to  receive  and  frequently  now  than  I  did  between  myself  and 

review  health  before.  clinicians  at 

information.  Conemaugh  or  VA. 

Strongly  Agree  Moderately  Agree  Strongly  Disagree  j  Moderately  Disagree  Not  Applicable 

Figure  69.  Provider  Satisfaction. 


The  graph  above  is  consistent  with  79%  (11/14)  being  in  agreement  that  they  are  more  satisfied  with  HIE 
exchange  via  NwHIN  compared  to  previous  routines,  e.g.  fax,  postal  mail,  etc.  (Q17).  Insight  into  this 
satisfaction  score  is  detailed  in  the  following  graph  asking  about  perceived  benefits  of  the  service.  Note, 
the  responses  to  the  perceived  benefits  under  consideration  above  were  not  mutually  exclusive. 
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Perceived  benefits  of  the  NwHIN/VLER  Exchange  were  as  follows: 


Q15  From  your  perspective,  what  are  the  benefits  of 
NwHIN/VLER  Exchange?  (Please  select  one  answer 
per  row)  Please  rate  how  much  benefit  you 
perceived  each  choice  to  have. 

Answered*  14  Skipped:  9 


100% 

80% 


50% 


40% 

20% 


0% 


L  Lt  ILL 


Care 

Improved 

Patient 

Reduced 

Less 

Reduced 

Coordination 

Decision 

Safety 

Unnecessary 

Burden  on 

Requests  to 

Making 

Tests 

Patients  to 

Medical 

Carry 

Records 

Record  Depts. 

Extremely  Beneficial 


Very  Beneficial 


Moderately  Beneficial  Q  Slightly  Beneficial  ^  Beneficial  at  all 


Figure  70.  Perceived  Benefits  of  eHealth  Exchange. 


The  perceived  benefits  shown  above  (Figure  70)  are  underscored  given  that  78%  agreed  that  HIE 
reduced  the  reordering  of  tests  (Q09);  and  57%  agreed  that  access  to  documents  from  remote  sites 
(enabled  by  HIE  via  NwHIN/VLER)  improved  their  ability  to  make  decisions  regarding  patient  care  -  36% 
replied  that  this  question  was  not  applicable  (Q08). 

Although  the  majority  of  providers  were  satisfied  (overall)  with  the  service  and  found  it  to  be  beneficial, 
fifty  percent  agreed  that  the  NwHIN/VLER  service  improved  their  productivity  while  43%  disagreed 
(Q07). 

Note,  the  responses  to  the  perceived  benefits  under  consideration  above  were  not  mutually  exclusive. 
Additional  cross  tab  analyses  are  located  in  Appendix  S. 

2.4.4  Data  Center  Performance  Metrics 

NGC  provided  a  sampling  of  monitoring  statistics  from  the  Glassfish  server  throughout  its  subcontract. 
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Glassfish  metrics  can  be  found  in  Table  42  and  Figure  71.  Although  not  continuous,  the  three  presented 
time  periods  represent  all  the  time  periods  received  from  NGC.  Noteworthy  is  the  nearly  doubling  of 
created  connections  with  a  concurrent  precipitous  drop  in  the  rate  of  severe  log  errors/uptime. 


Log  Start  Date 
Log  End  Date 

Log  Level  Error  =  SEVERE 
UPTIME,  days 

SEVERE  Log  Level  Errors/ 
(UPTIME,  days) 


Count  OVERflows 

Request  Load 

#  connections  created 

MAXtime  (bean-pool),  minutes 
Runtime- http- listener  1 
Runtime- http- listener  2 
Runtime- http- listener  3 


GF1 _ GF2 _ GF3 

12/19/2012  3/20/2013  7/23/2013 

1/ 16/20 13  5/8/ 2013  9/16/ 20 13 

129  48  1 

28,0  48.0  54.9 

4.60  1.0  0.02 

0  co  u  nt  0  co  u  nt  0  co  u  nt 


12062  Count  19250  Count  23640  Count 


1.3  3.1  S.5 

1.3  2.8  S.5 

0  0  0 


Table  42.  Glassfish  Metrics. 


Glassfish  Metrics 


=Count  of  Connections  Created 


•SEVERE  Log  Level  Errors/  ^^MAXtime  (bean-pool),  minutes 

(UPTIME,  days) 


Figure  71.  Glassfish  Metrics  by  Time  Period. 
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Protocol  A-16192.3  has  maintained  local  IRB  and  secondary  approval  status.  The  document  received 
initial  approval  from  MMC  Scientific  Review  Committee  (July  12, 2012)  and  local  CONEMAUGH  IRB 
approval  one  week  later  (July  17,  2012).  USAMRMC  IRB  approval  was  received  on  August  13, 1012. 

Protocol  A-16192.3  Institutional  Review  Board  Timeline 

•  Received  initial  Conemaugh  approval  of  protocol  (July  17,  2012) 

•  Received  initial  USAMRMC  HRPO  approval  of  protocol  (August  13,  2012) 

•  Received  Conemaugh  continuing  review  (July  16,  2013) 

•  Received  acknowledgement  of  continuing  review  receipt  by  USAMRMC  HRPO  (July  24,  2013) 

•  Received  Conemaugh  continuing  review  (July  8,  2014) 

•  Received  acknowledgement  of  continuing  review  receipt  by  USAMRMC  HRPO  (July  28,  2014) 

•  Received  Conemaugh  continuing  review  (July  7,  2015) 

•  Pending  acknowledgement  of  continuing  review  receipt  by  USAMRMC  HRPO  (submitted  by 
Conemaugh  July  28,  2015) 

Conclusion 

Concurrent  use  of  VA  and  non-VA  health  care  services  is  high  among  rural  veterans.  Of  the  9.1  million 
people  enrolled  in  VA  healthcare  benefits  (VHA  Office  of  Rural  Health).  3.2  million,  or  roughly  35%  live 
in  rural  communities  (Veterans  Health  Administration,  Support  Services  Center).  Over  20%  of  these 
enrollees  living  in  rural  areas  are  more  the  60  minutes  away  from  a  VA  Medical  Center  (over  40%  live 
more  than  90  minutes  away).  Given  existing  transportation  challenges  of  many  veterans  and  their  need 
for  multi-disciplinary  and/or  specialist  care,  health  information  exchange  between  VA  and  non-VA 
providers  is  essential  for  care  coordination  and  medical  co-management.  Non-VA  providers  generally 
view  the  VA  as  the  barrier  to  collaborative  care  and  information  sharing  (Gaglioti  A)(Lampman  2011)  and 
note  the  patient  is  the  main  vehicle  of  health  information  exchange  between  health  systems  and 
providers  (Nayar  P).  NwHIN,  now  eHealth  Exchange,  was  one  technically  driven  project  established  to 
create  shared  exchanges  of  health  information. 

At  the  conclusion  of  the  MIDHT  program,  Conemaugh  has  successfully  established  a  production 
exchange  of  live  patient  data  via  the  eHealth  exchange  as  well  as: 

•  Incorporated  data  domains  from  both  Allscripts  (Out  Patient)  and  McKesson  (In  Patient) 

Electronic  Health  Record  (EHR)  systems 

•  Provided  on  ONC  supported  pilot  for  electronic  patient  consent 

•  Contributed  code  products  to  the  open-source  community 

•  Achieved  CCHIT  accreditation/certification. 

In  support  of  the  eHealth  Exchange  mission,  the  MIDHT  program  has  established  a  "Gateway"  that 
serves  as  an  onramp  to  the  eHealth  Exchange  network. 

Though  MIDHT  addressed  barriers  to  communication  through  the  establishment  of  the  health  data 
exchange,  the  project  did  not  overcome  some  of  the  commonly  reported  problems  with  better  care 
coordination  between  VA  and  non-VA  providers.  Though  project  education  initiatives,  incentives, 
outreach,  and  systems  based  improvements  were  completed,  usage  by  the  Conemaugh  providers  was 
minimal.  Patient  and  provider  satisfaction  was  recognized,  however,  improved  care  coordination  and 
veteran  health  outcomes  sadly  failed  to  motivate  usage  within  our  own  health  system. 
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One  overall,  broad  based  note  on  the  MIDHT  project  implementation  that  proved  to  be  the  biggest 
obstacle  was  the  fact  that  HIE  is  not  a  primary  focus  of  our  organization  (Conemaugh).  Challenges 
therefore  existed  internally  to  redirect  system  priorities  away  from  patient  portals  and  general  operative 
system  exchanges  (i.e  e-prescribing,  notes,  etc)  within  our  IT  department.  Work  within  MIDHT  fell 
outside  the  scope  of  standard  daily  operations. 

Lessons  learned  are  not  dissimilar  to  those  learned  by  the  Department  of  Veterans  Affairs  (Bouhaddou 
0)(Byrne  CM)(Mueller  KJ).  Shared  lessons  include  overall  data  quality  and  interoperability  challenges 
(Botts  N),  specifically: 

•  The  vital  importance  of  workflow  engineering. 

•  Integration  of  electronic  health  records  is  time  intensive. 

•  Standards  specifications  and  best  practices  are  notwithstanding  and  this  inevitably  slows 
integration  progress.  There  is  a  direct  need  to  better  metrics  for  health  record  interoperability, 

•  Compliance  challenges,  as  Conemaugh  signed  the  DURSA,  but  it  was  also  necessary  that  project 
vendors  also  be  in  compliance. 

•  Privacy  issues  between  partnering  organizations  can  stifle  advancement 

•  The  involvement  of  integrating  new  systems  and  working  relationships  between  partners 
include  discussions  on  the  format,  content,  and  parameters  of  information  that  is  to  be 
exchanged. 

Like  other  reviews  of  HIE  (Rudin  RS),  our  stakeholders  found  value  in  health  information  exchange,  but 
barriers  of  acceptance  and  usage  existed.  Technical  and  workflow  issues  were  problematic,  but  in  the 
end,  these  were  more  easily  overcome  rather  than  stakeholder  engagement.  More  research  is  needed 
on  identifying  success  factors  and  operational  issues. 

Key  Research  Accomplishments 

•  Successfully  deployed  an  integrated  pharmacy  robotics  and  bar  code  medication  administration 
technology  platform  throughout  three  Conemaugh  hospitals 

•  Completed  an  extensive  research  analysis  of  the  impact  of  said  technology  on  medication  errors, 
provider  satisfaction,  workflow  and  financial  metrics 

•  Conemaugh  became  the  14th  Virtual  Lifetime  Electronic  Record  pilot  in  the  nation;  partnering 
with  the  Department  of  Veterans  Affairs  in  production  for  three  years 

•  Integrated  the  open  source  CONNECT  software  with  three  commercial  EHR  systems  utilizing 
significant  custom  build 

•  Enrolled  over  1,000  Veterans  in  the  program  for  health  information  exchange 

•  First  participant  in  the  nation  to  successfully  pass  new  "eHealth  Exchange"  testing  program 
(2010/2011  services) 

•  99%  patient  discovery  correlation  rate 


Reportable  Outcomes 

•  Presented  in  annual  research  poster  symposiums,  multiple  HIMMS  Interoperability  Showcases, 
CONNECT  Code-A-Thons,  patient  safety  fairs,  direct  and  indirect  community  outreach  efforts. 
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and  community  response  veterans  conferences  co-sponsored  by  the  Veterans  Community 
Initiatives  (VCI,  Johnstown,  PA;  http://www.vciinc.org) 

•  Made  multiple  CONNECT  code  contributions  to  TATRC,  Federal  Health  Architecture  and  Alembic 
Foundation 

Conclusion 

Conemaugh  Memorial  Medical  Center  has  completed  both  arms  of  the  MIDHT  project  within  the  POP. 
Statement  of  Work  (SOW)  tasks  were  all  successfully  executed.  Core  technologies  under 
investigation  included  pharmacy  robotics,  bar  code  medication  administration  (BCMA)  and  health 
information  exchange  via  the  eHealth  Exchange. 

Implementation  of  an  automated  pharmacy  robotics  system  and  complementary  bedside  BCMA 
technology  at  three  hospitals  was  a  significant  undertaking  that  directly  resulted  in  new  workflows 
for  many  healthcare  staff  and  valuable  lessons  learned  as  noted.  The  research  analysis  produced 
varied  results  with  an  overall  neutral  impact  on  medication  error  rates,  nursing  satisfaction  and 
provider  workflow. 

The  program  has  evolved  from  a  testbed  to  a  production  exchange  of  data  between  a  rural  health 
system  and  the  VA.  The  program  is  regarded  as  the  14th  VLER  pilot,  supports  meaningful  use  criteria, 
and  is  one  of  the  first  to  adopt  both  the  2010  and  2011  eHealth  exchange  specifications.  Though 
data  from  the  2009-2013  Electronic  Health  Records  Survey  concluded  the  electronic  health  record 
adoption  continues  to  develop  in  terms  of  usage  and  acceptance,  only  14%  share  data  with  providers 
outside  their  organization  (Furukawa  MF).  Conemaugh  was  pleased  to  establish  exchange  with  the 
James  E.  Van  Zandt  VA  Medical  Center.  As  a  major  healthcare  provider  for  veterans  in  southwestern 
Pennsylvania,  Conemaugh  needs  to  continue  to  develop  working  relationships  with  outside  providers 
to  enable  more  seamless  care  transitions  via  technology  and  electronic  health  records  systems. 

Lessons  learned  from  MIDHT  will  continue  to  shape  our  progress. 
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Appendix  A.  Lists  of  acronyms  and  abbreviations. 


Acronym 

Description/Definition 

API 

Application  Programming  Interface 

BCMA 

Bar  Code  Medication  Administration 

CAL 

Common  Access  Layer 

CHS 

Conemaugh  Health  System 

MMC 

Conemaugh  Memorial  Medical  Center 

COTS 

Commercial  Off  The  Shelf 

CPP 

Consumer  Preferences  Profile 

CVMH 

Conemaugh  Valley  Memorial  Hospital  (dba  "CONEMAUGH") 

DQ 

Document  Query 

DoD 

Department  of  Defense 

FHA 

Federal  Health  Architecture 

GUI 

Graphical  User  Interface 

HAPI 

HL7  Application  Programming  Interface 

HIE 

Health  Information  Exchange 

JIRA 

Bug  tracking  software 

JKS 

Java  KeyStore 

LDAP 

Lightweight  Directory  Access  Protocol 

MHS 

Military  Health  System 

MIMC 

Conemaugh  Miners  Medical  Center 

MIDHT 

Military  Interoperable  Digital  Hospital  Testbed 

MIS 

Management  Information  Systems 

MPI 

Master  Patient  Index 

MYMC 

Conemaugh  Meyersdale  Medical  Center 

NGC 

Northrop  Grumman  Corporation 

NIST 

National  Institute  of  Technology 

NwHIN 

Nationwide  Health  Information  Network  (now  called  "eHealth 
Exchange") 

OID 

Object  Identifier 

ONC 

Office  of  the  National  Coordinator  for  HIT 

OpenDS 

Open  Directory  Service 

PD 

Patient  Discovery 

PHI 

Protected  Health  Information 

QD 

Query  for  Documents 

RD 

Retrieve  Documents 

SAML 

Security  Access  Markup  Language 

SAN 

Storage  Area  Network 

SOW 

Statement  of  Work 

SSO 

Single  Sign  On 

SSL 

Secure  Socket  Layer 
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SVN 

Subversion 

TATRC 

Telemedicine  &  Advanced  Technology  Research  Center 

UC 

Universal  Client 

UDDI 

Universal  Description  Discovery  and  Integration 

USAMRMC 

United  States  Army  Medical  Research  and  Materiel  Command 

VA 

U.S.  Department  of  Veterans  Affairs 

VLER 

Virtual  Lifetime  Electronic  Record 

VPN 

Virtual  Private  Network 

WSDL 

Web  Services  Description  Language 

XDS 

Cross-Enterprise  Document  Sharing 
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Appendix  B.  Pharmacy  Robotics  Equipment  (MMC) 


Pharmacy  Robotics  Equipment 
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Appendix  C.  AdminRX  Analytics  Screenshots. 


McK  Adoption  Horizon  AdminRx  Analytics 


tity:  01 


Refresh 


Department:  Show  All  [▼] 
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Previous  5  Months 

Current 

Month 

Goal 

Achievement 

Nov 
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Mar  2013 

Medication  Administration 
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t 
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t 
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%  Med  Admins  w/Drug  Barcode  Scanned 

t 
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95.0%* 
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%  Med  Admins  Given  Early 

t 

t 

3.6%* 

4.1%* 
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4.3%* 

%  Med  Admins  Given  Late 

t 

t 
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16.7%* 

%  Admins  wNo  Med  Order  Found  Override 

t 

t 

iT%* 
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1.3%* 

1.3%* 

%  Admins  w /  Quantity  Override 

t 

t 
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0.7%* 

0.6%* 

%  Admins  w/No  Schedule  Found  Override 

t 

t 

1.2%* 

1.3%* 

1.4%* 

1.4%* 
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Medication  Administration  Warnings 

Total  #  of  Med  Admin  Warnings 

t 

t 
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126,397* 

112,451* 
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%  Med  Admin  Warnings  Overridden 

t 

t 
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48.3%* 

48.9%* 

47.3%* 

%  Med  Admin  Warnings  Leading  to  Modify 
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48.7%* 

48.5%* 

49.8%* 

%  Med  Admin  Warnings  Leading  to  Cancel 

t 

t 

52.4%* 

48.7%* 

48.5%* 
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IV  Administration 
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t 
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%  IV  Starts  w/  IV  Barcode  Scanned 

t 

t 

71.3%* 

63.3%* 

66.2%* 
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%  IV  Admins  w/  Wristband  Scanned 

t 

t 
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90.1%* 

90.2%* 

89.1%* 

- 

AdminRX  Analytics  Summary. 
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rmance  Analytics1 


Dashboard 


Highlights 


Reports 


Resources 


Display  •  Alerts  •  Drill  Up  •  Drill  Top  •  Export  •  Print  •  Preferences  •  Startup 


Scorecards 


Y  Logout  I 


McK  Adoption  HARx  Analytics  Medication  Administration 


Drill  1  of  8:  Month  Medication  was  Charted 

*  Drill  2  of  8:  Primary  Medication  Name;  from  Month  Medication  was  Charted  /  2013_03_March 
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AdminRX  Analytics  by  Medication. 


McK  Adoption  HARx  Analytics  Medication  Administration 

Drill  1  of  8;  Month  Medication  was  Charted 
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33 
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13 
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22 
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Total 
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AdminRX  Analytics  by  Unit. 
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Appendix  D.  Pharmacy  Robotic  Dispensing  Survey. 


Pharmacy  Survey  -  Robotic  Dispensing 


Thank  you  for  taking  the  time  to  complete  this  anonymous  survey.  We  are  interested  in 
your  responses  about  how  the  McKesson  Robot-RX®  system  has  impacted  your  delivery 
of  patient  care.  By  completing  and  submitting  this  evaluation  form,  you  are  consenting  to 
take  part  in  this  study. 

Instructions: 

Please  read  each  item  carefully  and  rate  the  technology  provided  to  you  for  this  research 
study.  If  you  need  assistance  completing  this  form,  please  contact  John  Hargreaves  at 
(814)  269-5277. 

1.  From  your  perspective,  the  McKesson  Robot-RX®  system  has  increased  filling 
accuracy. 

Strongly  Agree  Agree  Neutral  Disagree  Strongly  Disagree 

2.  From  your  perspective,  the  McKesson  Robot-RX®  system  has  significantly 
reduced  missing  medications. 

O  Strongly  Agree  O  Agree  o  Neutral  o  Disagree  Strongly  Disagree 


3.  From  your  perspective,  the  McKesson  Robot-RX®  system  has  minimized  issues 
with  expired  medications. 


O  Strongly  Agree 


o 


Agree 


o 


Neutral 


Disagree 


Strongly  Disagree 


4.  From  your  perspective,  the  McKesson  Robot-RX®  system  has  significantly 
reduced  the  amount  of  checking  time  needed  and  medications  are  delivered  to  the 
floor  quicker. 


o 


Strongly  Agree 


Agree 


o 


Neutral 


o 


Disagree 


o 


Strongly  Disagree 


5.  How  would  you  rate  the  Robot-Rx®  system  and  associated  components  with 
respect  to  ease  of  use. 

O  Very  Easy  Q  EaSy  O  Neutral  O  Difficult  Very  Difficult 


6.  What  percentage  of  the  hospital’s  medications  has  been  impacted  by  this  system? 

O  100%  Q  95-99%  o  90-94%  Q  85-89%  o  80-84%  o  <80%  o  Not  Sure 


7.  You  have  been  able  to  spend  more  time  on  clinical  activities  since  the  new  system 
was  implemented? 

True  False  N/A 


8.  If  true,  how  many  minutes  per  day? 
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Pharmacy  Survey  -  Robotic  Dispensing 


9.  From  your  perspective,  the  McKesson  Robot-RX®  system  has  improved  patient 
safety  through  a  reduction  in  “near  misses”  and  reportable  medication  errors. 

o  Strongly  Agree  O  A9ree  o  Neutral  Disagree  Q  Strongly  Disagree 

10.  Would  you  recommend  this  system  to  other  hospitals? 

Yes  No  Not  Sure 

1 1 .  How  long  have  you  been  a  pharmacist  or  technician? 

O  0-5  Years  o  6-10  Years  o  11-15  Years  o  16-20  Years  o  20+  Years 

12.  Please  provide  additional  comments  on  this  topic  as  necessary. 

I  il 
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Appendix  E.  Nurses'  Survey  -  Medication  Administration  System  Survey. 


Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 

1 .  Welcome 


Thank  you  for  participating  in  the  Medication  Administration  System  Survey  — 

Nurses1  Assessment  Survey  (MAS-NAS-)  project. 

Completing  this  form  indicates  you  recognize  that  this  is  a  research  project.  As  a  token  of  our  appreciation,  you  will  receive  a  $20.00  incentive  from 
Sheetz  when  you  have  completed  the  survey.  After  completion ,  please  call  S14’269'5277  to  arrange  receipt  and  please  have  your  “SUM’ 
available.  We  reserve  the  right  to  withhold  incentive  for  any  reason  related  but  not  limited  to  that  which  compromises  the  survey  intent,  data 
validity,  completion  compliance,  or  respondent  eligibility. 

If  you  previously  completed  a  MAS-NAS,  please  complete  this  survey  so  we  can  lean  how  barcode/eMAR  has  changed  how  you  administer 
medications.  We  will  compare  nurses"  responses  on  this  MAS-NAS  with  previous  surveys  completed  in  2012  to  learn  the  impact  of  convening  to 
bsrcode/eMAR  on  CHS  nurses. 

If  this  is  your  first  MAS-NAS,  we  would  like  to  learn  your  beliefs  about  the  system 

Please  create  a  five  digit  confidential  identifier  called  SUM.  This  allows  us  to  maintain  your  privacy  while  being  able  to  compare  surveys  taken  at 
different  times.  Even  if  this  is  your  first  survey,  please  compute  your  SUM.  We  ask  you  to  calculate  the  following  number  and  place  it  in  the  space 
provided.  The  number  is  the  SUM  of  the  last  five  digits  of  your  social  security  number  added  to  the  last  five  digits  of  your  home  telephone  number. 
If  your  telephone  number  changed,  please  use  the  number  you  previously  used.  Only  the  SUM  of  those  two  numbers  should  be  recorded  on  the 
first  page  of  the  survey.  If  your  sum  has  more  than  five  digits,  record  only  the  last  five.  See  example  below: 

Last  five  social  security  numbers  =  52346 
+ 

Last  five  home  telephone  numbers  =  58721 
SUM  =  111067 

ENTER  11067  (last  five) 

After  calculating  your  SUM.  please: 

Add  your  SUM  an  Ihe  next  page  and  complete  the  survey. 

Thank  you  for  your  participation  in  this  important  project! 


2s  Introduction 


*1.  SUM: 


*2.  Date  of  completing  the  survey: 
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*3.  Nursing  Unit: 

Meyersdale  Medical  Center 
Miners  Medical  Center 
Ashman  6  ICU 
Ashman  7 
Ashman  8 
Ashman  9 
Ashman  10 
Rose  6  ICU 
Rose  7  CICU 
Rose  7  Telemetry 
Rose  8 
Rose  8  PCU 
Rose  9 
Rose  10 
E4 
E6 
GS4 
GS  5 
GS  6 
ASU 
PACU 

Crichton  Rehab 
Med  Surg  7 
RICN 
Maternity 
Pediatrics 
Geropysch 
Aloysia  Hall 
Behavioral  Health 
School  of  Nursing 
Float 
TCU 
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Other  (please  specify) 


The  following  statements  describe  experiences  and  beliefs  about  the  current  medication  administration  system  you  use.  By  current  medication 
administration  system,  we  mean  the  institutional  structures  and  operational  policies  that  direct  and  support  the  process  and  procedures  of 
delivering/administering  pharmaceutical  products  to  patients.  This  includes  all  activities  from  medication  ordering  to  dispensing  to  administration 
and  documentation  as  well  as  all  supplies  and  technologies  associated  with  the  current  medication  system,  such  as  computer  terminals,  medication 
storage  units,  drug/drug  interaction  information,  patient  education  sheets,  infusion  pumps,  tubings,  syringes,  etc  that  are  required  in  the  process  of 
getting  medications  to  patients. 

Completing  this  survey  should  take  about  10  minutes. 

By  completing  this  survey,  you  are  indicating  consent  to  participate  in  this  evaluation. 

Please  read  each  statement  and  circle  the  number  that  best  expresses  your  own  experiences  and  beliefs. 

There  are  no  right  or  wrong  answers. 

Do  not  write  your  name  on  the  survey. 

Please  indicate  the  degree  to  which  you  agree  or  disagree  with  each  item  by  using  the  following  RATING  SCALE. 

Choose: 

1  if  you  strongly  agree  with  the  statement. 

2  if  you  moderately  agree  with  the  statement. 

3  if  you  slightly  agree  with  the  statement 

4  if  you  slightly  disagree  with  the  statement. 

5  if  you  moderately  disagree  with  the  statement. 

6  if  you  strongly  disagree  with  the  statement 
NA  if  the  statement  does  not  apply  to  you. 

Example:  The  current  medication  administration  system  provides  adequate  information  about  possible  mediation  side  effects. 

Answer:  If  you  believe  that  you  always  have  access  to  the  type  and  amount  of  information  you  need  to  be  able  to  assess  for  and  manage  potential 
side  effects  of  medications  you  administer,  you  should  circle  1  because  that  statement  best  expresses  your  belief. 


3.  Questions 


4.  Because  of  information  available  through  the 
current  medication  administration  system  I  know  both 
the  intended  actions  and  side  effects  of  medications  I 
administer. 


O’ 

O’ 

O’ 

o« 

O’ 

O’ 

O  na 

Strongly 

Moderately 

Slightly 

Slightly 

Moderately 

Strongly 

Not 

Agree 

Agree 

Agree 

Disagree 

Disagree 

Disagree 

Applicable 

5. 1  find  the  drug  alert  feature  (drug/drug  or  drug/food 
interaction)  of  the  current  medication  administration 
system  helpful. 


O’ 

O’ 

O’ 

O’ 

O’ 

<o 

o 

o 

z 

> 

Strongly 

Moderately 

Slightly 

Slightly 

Moderately 

Strongly 

Not 

Agree 

Agree 

Agree 

Disagree 

Disagree 

Disagree 

Applicable 
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6.  The  current  medication  administration  system 
makes  it  easy  to  check  active  medication  orders 
before  administering  medications. 

O1  O 2  O  O  O5  O 6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

7.  The  current  medication  administration  system 
provides  me  with  information  to  know  that  a 
medication  order  has  been  checked  by  a  pharmacist 
before  I  administer  the  medication. 

O1  O2  O3  O  O6  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

8.  The  current  medication  administration  system 
promotes  2-way  communication  between  clinicians 
(MD,  Pharmacist,  RN)  about  medication  orders. 

0 1  O2  O3  O  O6  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 


9. 1  have  access  to  the  systems  that  support 
medication  administration  (physician’s  orders,  drug 
information)  when  I  need  them. 

O1  O2  O 3  O 4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

10.  The  drug  information  available  through  the  current 
medication  administration  system  is  easy  to  get  when 
I  need  that  information. 

O1  O2  O3  O  O 5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 


123 


Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 


11.1  know  where  all  the  medications  I  need  are  stored 
(either  on  the  unit  or  if  they  need  to  be  procured  from 
the  pharmacy). 

O1  O2  O3  O  O5  O 6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

12.  The  current  medication  administration  system 
helps  me  to  be  efficient  at  medication  administration. 

O'  O2  O3  O4  O5  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

13.  The  current  medication  administration  system 
makes  it  easy  to  check  that  I  am  following  the  “5 
rights’9  when  I  administer  medications. 

O1  O2  03  04  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

14.  The  turnaround  time  for  receiving  medications 
needed  “stat”  or  for  patients  newly  admitted  to  the 
unit  is  adequate. 

0 1  O2  03  04  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

15.  The  current  medication  administration  system  is 
effective  in  reducing  and  preventing  medication 
errors. 

O1  O2  O3  O4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

16.  The  current  medication  administration  system  is 
user-friendly  to  the  nurses  who  administer 
medications. 

Eh  Eh  Eh  Eh  Eh  Eh  EIna 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 
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17.  The  equipment  and/or  supplies  needed  to 
administer  medications  are  readily  available  to  me. 

O1  O 2  O3  O  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

18.  Information  available  through  the  current 
medication  administration  system  helps  me  to  know 
what  to  do  should  my  patient  have  any  bad  reactions 
from  a  medication. 

o- 


O’  O2 

Strongly  Moderately 
Agree  Agree 


O’ 

Slightly 

Agree 


o 

Slightly 

Disagree 


o 

Moderately 

Disagree 


o 

Strongly  Not 
Disagree  Applicable 


19. 1  have  to  keep  stashes  of  medications  to  be  sure  I 
have  medications  I  need  when  I  need  them. 

O’  O2  O 3  O 4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 


5.  Open-ended 


20.  Please  add  any  comments  you  wish  about  the  current  medication  administration 
system  and  the  degree  to  which  components  of  the  current  system  support  your  ability  to 
administer  medications  safely  and  professionally. 


21.  If  you  could  change  one  thing  in  the  current  medication  administration  system,  what 
would  it  be? 


3 


22.  Did  you  work  on  an  inpatient  nursing  unit  at  CHS  before  BCMA  ( Admin-Rx)  was 
implemented? 

o-  o 
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Comparing  now  (using  barcode/eMAR)  with  the  old  system,  please  respond  to  the  following  7  statements. 

23.  It  is  easier  to  do  all  the  checking  steps  needed  during  the  medication  administration 

process. 

Strongly  Moderately 

Agree  Agree 

Slightly  Slightly 

Agree  Disagree 

Moderately  Strongly  Did  not  use 

Disagree  Disagree  old  system 

24.  This  is  a  safer  system  for  patients. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

25.  With  the  new  system,  it  is  easier  to  access  information  1  need  to  administer 

medications. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

26. 1  am  more  satisfied  with  this  new  medication  administration  system  than  with  the 

previous  one. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

27. 1  have  more  time  to  spend  with  patients. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

28.  Barcode/eMAR  has  made  the  medication  administration  process  more  efficient  for  me. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 
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29.  Medications  are  more  readily  available  when  I  need  them  for  patients. 


Strongly  Agree 
Moderately  Agree 
Slightly  Agree 


Slightly  Disagree  Did  not  use  old  system 

Moderately  Disagree 
Strongly  Disagree 


8.  Closing 


*30.  Overall,  how  satisfied  are  you  with  the  current  medication  administration 
system?  Please  select. 

O1-  O2  O3  O4  O5-  O®  O7  O8  O9  O10- 

Completely  Neither  Completely 

Dissatisfied  satisfied  Satisfied 


dissatisfied 


*31.  How  long  have  you  been  using  the  BCMA  system  (Admin-Rx)  with  patients? 

o  Less  than  1  month  o  4-6  months  o  10-12  months 

o  1*3  months  o  7-9  months 


Please  provide  the  following  information  about  yourself  and  your  background 


32.  Gender: 

Q  Male 


o 


Female 


33.  Age: 


34.  Highest  nursing  degree: 

LPN  Diploma 

Q  AS/AD 

Q  BS/BSN 

Q  MS/MSN 

35.  Number  of  years  employed  as  a  nurse: 


*36.  Number  of  years  employed  byConemaugh: 


37.  Number  of  hours  worked  in  a  typical  week: 
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38.  Typical  shift  rotation  schedule: 

Q  All  Shins  Q  Rotate  D/E  Q  7A-7P 

Evenings  o  Rotate  D/N  o  7P-7A 

Nights  All  Days 


39.  Typical  weekly  schedule: 

mostly  weekend/holiday 
mostly  Monday-Friday 


*40.  Current  position: 

staff/direct  patient  care  (staff  RN/NIC) 
leadership/management 
Other  (please  specify) 


rotate  weekdays/weekends/holidays 


o  education 
o  Student 


41.  Compared  to  your  nursing  peers,  how  do  you  rate  your  computer  skills? 


Above  Average 


o 


Average 


o 


Below  Average 


42.  Do  you  use  a  computer  at  home? 

O Yes  o° 


43.  How  do  your  rate  your  skill  at  obtaining  patient  care  information  from  the  Conemaugh 
computer  systems? 

o  Excellent  o  Good  o  Fair  o  Poor 


44.  Did  you  ever  use  barcode/eMAR  in  another  hospital  before  working  at  CHS? 

O Yes  O 

Copyright  by  Ann  C.  Hurley,  Diane  R.  Lancaster,  Judy  Hayes,  Anne  Bane.  Chantel  Wilson  Chase.  2003.  Do  not  copy,  circulate,  or  cite  without 
written  permission  of  the  researchers. 
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Appendix  F.  Nursing  Student  -  Medication  Administration  System  Survey. 


Nursing  Student  BCMA 


This  survey  is  intended  for  nursing  students  enrolled  in  the  Conemaugh  School  of  Nursing.  We  are  asking  for  your  feedback  regarding  bar  code 
medication  administration  (BCMA)  (Admin  Rx)  at  Conemaugh  Memorial  Medical  Center.  As  a  token  of  our  appreciation,  you  will  receive  a  $20  00 
incentive  from  Crossroads  cate  when  you  have  completed  the  survey  After  completion,  please  call  814-269-5232  to  arrange  receipt  and  please 
have  your  “CODE'  available  We  reserve  the  right  to  withhold  incentive  for  any  reason  related  but  not  limited  to  that  which  compromises  the  survey 
intent,  data  validity,  completion  compliance,  or  respondent  eligibility.  Thank  you  for  your  time  and  input. 

*1.  Please  create  a  7  character  "CODE"  using  both  numbers  and  letters.  Enter  here. 

I  I 

*2.  Please  state  your  gender: 

O Ma,e  o  Female 

*3.  Please  select  your  age: 

Round  your  age  to  the  nearest  whole  year. 

20  years  or  less  31  -  40 

o  21-30  o  41  -50 

*4.  Today's  date: 

I  1 

*5.  Current  status: 

O  1st  Year  Sludent  O  2nd  Year  Student 

*6.  How  often  do  you  use  the  medication  carts  (blue  Rubbermaid  with  drawers)  on 
inpatient  units? 

Frequently  Very  Often  Sometimes  Not  Very  Often  Never 

*7.  Do  you  have  access  to  medication  carts  (blue  Rubbermaid  with  drawers)  when  you 
need  them  for  training  on  the  floor? 

Frequently  Very  Often  Sometimes  Not  Very  Often  Never 

*8.  The  current  medication  administration  system  helps  me  to  be  efficient  at  medication 
administration. 

Strongly  Agree  Slightly  Disagree 

Moderately  Agree  Moderately  Disagree 

Slightly  Agree  Strongly  Disagree 


Q  51  -  60 

Greater  than  61  years 


Page  1 


129 


Nursing  Student  BCMA 


*9.  The  current  medication  administration  system  makes  it  easy  to  check  that  I  am 
following  the  “5  rights”  when  I  administer  medications. 


Strongly  Agree 
Moderately  Agree 
Slightly  Agree 


Slightly  Disagree 
Moderately  Disagree 
Strongly  Disagree 


*10.  The  current  medication  administration  system  is  effective  in  reducing  and 
preventing  medication  errors. 


Strongly  Agree 
Moderately  Agree 
Slightly  Agree 


Slightly  Disagree 
Moderately  Disagree 
Strongly  Disagree 


*11.  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 


O-  O'  O2  O3  O4  O5-  Oe  O7  Os 

Completely  Neutral 

Dissatisfied 


O8  O10- 

Completely 

Satisfied 


*  12.  Do  you  have  previous  BCMA  experience  outside  of  Conemaugh? 

O Yes  o 


13.  Please  add  any  comments  you  wish  about  the  current  medication  administration 
system. 

3 

Thank  you  for  your  time  and  input!  Have  a  good  day. 


Page  2 
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Appendix  G.  Physician  Medication  Administration  Survey. 


Physician  -  BCMA  and  eMAR  System 


Thank  you  for  taking  the  time  to  complete  this  anonymous  survey.  We  are  interested  in 
your  responses  about  how  the  new  Bar  Code  Medication  Administration  (BCMA)  and 
electronic  Medication  Administration  Record  (eMAR)  system  has  impacted  your  delivery 
of  patient  care.  By  completing  and  submitting  this  survey,  you  are  consenting  to  take  part 
in  this  study. 

Instructions: 

Please  read  each  item  carefully  and  rate  the  technology  provided  to  you  for  this  research 
study.  If  you  need  assistance  completing  this  survey,  please  contact  John  Hargreaves  at 
(814)  269-5277. 

1.  How  often  do  you  access  the  electronic  medication  administration  record  (eMAR) 
module  available  through  Care  Portal? 

O  Frequently 

o  Sometimes 
o  Seldom 
o  Never 

2.  The  new  eMAR  module  has  improved  my  decision-making  and  efficiency  because 
I  can  now  electronically  access  if/when  ordered  medications  were  administered 
from  various  locations. 

o  Strongly  Agree 

O  Agree 

Neutral 

o  Disagree 

Strongly  Disagree 

3.  Adverse  drug  events  have  reduced  because  of  the  bar  coding  technology  and 
additional  nursing  safeguard  warnings/alerts  in  place  to  reduce  medication  errors. 

O  Strongly  Agree 
O  Agree 
Neutral 

o  Disagree 

Strongly  Disagree 

o  Don't  Know 
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4.  The  new  Bar  Code  Administration  Administration  (BCMA)  system  and  Care  Portal  eMAR 
module  have  improved  physician  communication  with  pharmacy  and  nursing. 

O Yes 

o° 

Not  Sure 


5. 1  would  recommend  our  new  BCMA  system  and  Care  Portal  eMAR  module  to  other  acute 
care  hospitals. 

Strongly  Agree 

O  Agree 

Neutral 

o  Disagree 

Strongly  Disagree 


6.  How  many  years  have  you  been  a  practicing  physician  or  physician  assistant? 


Not  Applicable 
0-5  Years 
o  6-10  Years 
o  11-15  Years 
o  1 6-20  Years 
o  20+  Years 


7.  Please  select  your  affiliation. 


o 

o 

o 

o 

o 


Physician  -  Conemaugh  Physician  Group 

Physician  -  Hospitalist 

Physician  -  Independent 

Physician  Assistant 

Resident 


8.  Do  you  have  BCMA  and  eMAR  experience  outside  of  Conemaugh? 

O- 

o 


9.  Please  provide  comments  on  this  topic  as  necessary. 
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Appendix  H.  Selected  Software  Development  Activities,  including  vendors  NCG  and  CGI  Federal,  Inc. 


2010 

1.  A  "Kick  Off  Meeting"  was  held  with  TATRC  representatives  on  October  19,  2010  to  review 
project  deliverables.  A  decision  was  made  to  proceed  using  the  CONNECT  version  3.1 
architecture  for  health  information  exchange  (HIE)  in  a  production  environment. 

2.  Adrian  Anderson  (NHIN  operations  team)  provided  the  NHIN  conformance  testing  packet  on 
November  24,  2010  to  Conemaugh.  These  tests  needed  executed  during  the  NHIN  on-boarding 
process. 

3.  The  CONNECT  v3.1  document  assembler  plugin  has  been  "patched"  to  create  a  C32  document. 
As  such,  the  code  was  shared  with  the  TATRC  Advanced  Concepts  Team  (ACT)  for  review  and 
use  on  December  17,  2010.  Conemaugh  also  provided  the  code  to  the  Federal  Health 
Architecture  (FHA)  CONNECT  team  for  incorporation  into  their  baseline/ 

4.  Conemaugh  led  a  MIDHT  demo  at  the  HIMSS  Interoperability  Showcase™  in  Orlando,  FL  in 
February  2011. 

2011 

1.  Implemented  document  assembler  changes  for  onset  date  within  Allergies  and  Problemst 

-  Updated  Document  Assembler  to  populate  the  <text>  field  in  the  C32  with  the  parsed  <text> 
field  from  the  CareRecord  for  Allergies  and  Problems  (parsed  the  date  information  from  the 
text  field). 

-  Updated  the  Conemaugh  style  sheet  to  display  the  <text>  field  if  the  effectiveTime  field  was 
not  populated. 

2.  Resolved  the  persistence  issues  within  the  patient  consent  module  v3.1+ 

-  Opt-in/Opt-out  consumer  consent  choices  are  now  persisted. 

3.  Created  WSDL  for  Emergency  Room  Discharge  Summaryt 

-  The  CAL  WSDL  was  updated  and  provided  to  Conemaugh  as  part  of  the  overarching 
Emergency  Room  (ER)  Discharge  Summary  tasking. 

4.  CONNECTUniversalClientGUI  SAMLerrors+ 

-  This  update  addresses  the  issue  that  is  encountered  when  the  user  tries  to  search  for  a 
patient  from  the  mpi.xml  file  using  the  CONNECTUniversalClientGUI  application.  Specifically, 
adding  the  appropriate  xml  files  to  the  src\main\java\META-INF  folder  and  updating  the 
wsit-client.xml  file  to  reflect  that  the  missing  files  were  added  solves  this  issue. 

5.  Multiple  items  returned  from  Initiate* 

-  Modified  the  Initiate  Connector  to  parse  the  Initiate  response  in  the  event  that  multiple 
patients  are  returned  with  the  same  EID.  Response  returned  contains  the  most  updated 
patient  entry  per  EID. 

6.  Updates  to  CONNECT  Universal  Client  GUI+ 

-  This  update  allows  the  user  to  choose  targeted  gateways  via  a  combo  box  that  appears 
under  both  the  patient  discovery  and  document  query  tabs.  This  update  also 
dynamically  chooses  a  style  sheet  to  render  a  document  against,  based  on  the  OID  of  the 
responding  gateway. 


*  Completed  by  Northrop  Grumman 
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7.  Implemented  Spring  Injection  points  in  CONNECT  v3.1  framework  for  Patient  Discoveryt 

-  Spring  Injection  points  now  allow  for  one  of  four  selectable  implementations  for  every 
service  in  CONNECT  v3.1  framework  for  Patient  Discovery 

8.  Patient  Discovery  -  Responding  side  to  Initiatet 

-  Allows  a  patient  discovery  request  from  a  remote  gateway  to  pass  through  the  CHS 
Gateway  and  adapter,  then  on  to  Initiate,  where  a  patient  lookup  is  performed,  and  the 
results  are  passed  along  to  the  requestor.  This  external  search  allows  at  most  one 
patient,  and  a  high  threshold  (minScore)  to  be  sent  to  Initiate.  Also,  internal  patient 
search  has  been  enhanced  to  use  the  same  flow  through,  with  different  rules  (allows 
multiple  returns,  sets  different  min  score  (lower  threshold)  to  Initiate) 

9.  Patient  Discovery  -  Initiating  side  to  Initiatet 

-  Changes  were  made  to  the  CONNECTUniversalClientGUI  in  order  to  facilitate  the  building  of 
a  Patient  Discovery  Request  that  can  be  sent  to  all  configured  gateways  or  to  targeted 
gateways.  In  support  of  the  target  gateway  functionality,  a  textbox  was  added  under  the 
Patient  Discovery  tab.  This  textbox  allows  users  to  input  the  OIDs  of  target  communities.  In 
addition  to  the  updates  that  were  made  to  the  GUI  code,  updates  were  also  made  to  the 
interactionld  value  that  is  contained  within  the  CHSCoreLib  and  CONNECTCoreLib  projects. 
The  change  updates  the  value  to  reflect  the  actual  type  (PRPAJN201305UV02  instead  of 
PRPAJN201305UV)  of  the  request  that  is  being  sent  out. 

10.  Modified  Patient  Discovery  Response  to  be  compliant  with  Patient  Discovery 

Specification! 

-  This  modification  enhanced  the  response  created  by  the  Initiate  Connector  by  making  it 
Patient  Discovery  schema  compliant 

11.  Added  functionality  to  the  Document  Assembler  to  allow  for  handling  of  multiple 
Emergency  Room  Discharge  Summaries  (C62's)f 

-  Enhanced  the  Document  Assembler  to  be  able  to  assemble  (query  and  retrieve)  multiple 
C62  documents,  for  a  single  patient. 

12.  Updated  CONNECT  Universal  Client  GUI  Inbox  to  allow  for  display  C62's+ 

-  The  existing  VLER  Inbox  code  has  been  incorporated  into  the 
CONNECTUniversalClientGUI  so  that  C62documents  can  be  rendered. 

13.  C62  documents  are  not  returned  during  a  query  if  a  C32  document  is  requested  (based  on 
class  code)+ 

-  Code  modified  to  enforce  class  code  sent  in  request  for  all  applicable  documents. 

14.  Investigated  enforcement  of  Opt-ln/Opt-Out  for  Patient  Consent! 

-  When  Patient  Consent  is  configured  to  support  the  Opt-ln/Opt-Out  enforcement,  this  code 
will  block  all  requests  for  data  for  any  patient  who  has  "Opted  Out"  or  who  has  not  made 
an  Opt-ln  or  Opt-Out  selection.  Data  will  be  returned  only  for  those  patients  who  have  made 
an  Opt-ln  choice  and  had  their  choice  stored  in  the  document  database  in  their  Consumer 
Preferences  Profile  (CPP)  document. 

15.  Demonstrated  the  ability  to  secure  the  MIDHT  GUI's  using  OpenSSO  in  conjunction  with  OpenDS 
and  document  the  installation  procedures.! 

16.  On  October  21,  2011,  Allen  Barger  and  Zeke  Bravo  participated  in  a  demonstration  of  the 
Consumer  Preferences  Profile  (CPP)  GUI  in  which  the  opt-in  and  opt-out  capabilities  were 
demonstrated.! 


+  Completed  by  Northrop  Grumman 
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1.  Changes  were  made  by  Conemaugh  developers  to  the  FindDocumentWithContent  endpoint  in 
order  to  add  inpatient  discharge  summaries  from  three  Conemaugh  hospitals.  VA  was  able  to 
successfully  query/retrieve  a  discharge  summary  on  March  20,  2012. 

2.  In  order  to  spur  adoption  of  the  NwHIN  service  by  Conemaugh  providers,  technical  teams  have 
integrated  the  CONNECT  UniversalClientGUI  with  the  existing  McKesson  Care  Portal.  A  new  tab 
has  been  created  and  enrolled  users  will  be  able  to  access  NwHIN  data  from  an  application  that 
is  used  heavily  on  a  daily  basis.  Context  sharing  has  been  implemented  so  the  provider  does  not 
have  to  duplicate  the  patient  search. 

3.  Integration  and  testing  of  production  MPI  and  clinical  systems  via  CAL  occurred  during  the 
month  of  April  2012.  Issues  identified  during  testing  were  quickly  resolved  by  the  Conemaugh 
development  team.  Project  team  confirmed  that  all  C32  data  (problems,  allergies,  medications) 
is  active  to  the  patient  and  removed  the  default  order  date  for  "recorded"  medications. 

4.  Conemaugh  decided  to  add  radiology  reports  (C62)  from  various  hospital  facilities  to  the 
production  CAL  service.  Development  work  was  completed  on  August  10,  2012,  which  included 
all  radiology  modalities  (e.g.  x-ray,  CT,  etc.)  from  the  McKesson  Horizon  Patient  Folder  system. 
Consequently,  an  update  was  provided  to  the  VA  team  and  they  quickly  confirmed  successful 
receipt  of  the  said  data. 

5.  Provided  the  Conemaugh  technical  team  with  instructions  on  how  to  implement  2-way  SSL  on 
Apache  Tomcat  and  Microsoft  IIS  application  servers/ 

6.  Complied  with  the  VA's  User  acceptance  Testing  (UAT)  by  implementing  and  testing  OID  and 
URLs  on  the  MIDHT  production  (VA  testing)  server.! 

7.  Conemaugh  testing  of  medications  data  by  providing  log  file  snippets  that  contain  a  response 
from  the  Common  Access  Layer  (CAL)  medications  endpoint.f 

8.  Used  the  Lantana  testing  tool  to  validate  the  structure  of  Conemaugh's  Radiology  Studies 
documents. + 

9.  Modified  the  C62  document  builder  code  to  build  Radiology  Studies  documents  from 
information  provided  by  Conemaugh's  CAL  web  service  interface.! 

Donated  the  MIDHT  CONNECT  Universal  Client  GUI  code  to  FHAt: 

•  Created  a  virtual  machine,  running  CONNECT  v3.3.1,  that  can  be  used  to  test  MIDHT  code 
donations 

•  Prepared  a  Release  Notes  document  for  the  CONNECT  UC  GUI  code  donation 

•  Upgraded  the  MIDHT  CONNECT  UC  GUI  to  work  with  CONNECT  v3.3.1 

•  Staged,  committed  and  pushed  code  changes  to  the  MIDHT  Fork  in  GitHub 

•  Submitted  a  Pull  Request  from  within  GitHub 

•  Updated  the  TATRC  Subversion  repository  with  the  revised  GUI  code 

•  Installed  the  CONNECT  formatting  templates  for  the  NetBeans  IDE 

Donated  the  MIDHT  Common  Access  Layer  (CAL)  code  to  FHAt: 

•  Cleaned  up  the  AdapterCommonDataLayerEJB  project  by  removing  all  of  the  deprecated  files 
and  "dead"  code 


+  Completed  by  Northrop  Grumman 
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•  Prepared  release  notes,  change  list  and  install  instructions  documentation 

•  Upgraded  the  AdapterCommonDataLayerEJB  project  to  work  with  CONNECT  v3.3.1 

•  Staged,  committed  and  pushed  the  code  changes  to  the  MIDHT  Fork  in  GitHub 

•  Submitted  a  Pull  Request  to  the  CONNECT  3.3_integration  branch 

•  Updated  the  TATRC  Subversion  repository  with  the  revised  code 

•  Installed  and  applied  the  CONNECT  formatting  templates  for  the  NetBeans  IDE 

•  Created  a  SoapUl  project  that  can  be  used  to  test  the  CAL  donation 

•  Removed  all  Conemaugh-specific  identifiers  from  the  donated  code  and  configuration  files. 


2013 

1.  Modified  the  VA  style  sheet  so  that  the  date  values  in  the  Lab  Results  section  are  formatted  and 
the  header  of  each  C32  section  contains  information  about  the  patient  (name,  date  of  birth). 

2.  Appended  "(INCLUDES  LAB  RESULTS)"  to  the  end  of  the  "34133-9  Summarization  of  Episode 
Note"  entry  in  the  CONNECT  Universal  Client  GUI's  Select  Document  Types  drop-down  box. 

3.  Made  the  following  changes  in  production  to  the  Medications  section  of  the  Conemaugh  C32 
document  per  request  from  VA: 

-  Populated  the  substanceAdministration  text  tags  with  only  the  medication  Sig  value 

-  Put  the  section  text  <content>  tags  in  the  same  order  as  the  corresponding 
SubstanceAdministration  entries 

-  Changed  the  ID  attribute  of  each  section  <content>  tag  to  be  the  medication  Sig  value 
instead  of  the  name  of  the  medication 

4.  Created  user  accounts  in  OpenDS  and  updated  passwords  as  requested. 

5.  When  requested,  queried  the  MIDHT  Audit  Repository  to  confirm  the  Retrieve  Document 
actions  of  certain  VA  users. 

-  In  response  to  the  Healtheway  eHealth  Exchange  support  staff  assuming  administrative 
control  for  the  UDDI  domain  registrar  services,  connectivity  from  the  production 
Conemaugh  gateway  to  the  production  eHealth  Exchange  UDDI  was  tested  on  July  8,  2013. 

6.  Patient  consent  issue 

-  Patient  consent  documents  were  not  being  saved  to  the  document  repository  because  of  a 
bug  in  CONNECT,  which  used  an  inappropriate  method  for  assigning  primary  keys.  The  bug 
fix  was  developed,  deployed,  and  verified  with  Conemaugh,  and  then  applied  to  the 
CONNECT  trunk. 

TATRC  Partners  and  Open  Source  Community  Outreach 

•  NGC  became  a  regular  participant  in  the  eHealth  Exchange  Spec  Factory  all-hands  weekly 
teleconference. 

•  COGON  Systems  reached  out  to  Allen  Barger  for  help  with  validating  the  C62  document  that 
they  had  been  sending  to  the  VA.  Allen  helped  COGON  validate  their  C62  document  by 
performing  the  following  actions: 

-  Compared  the  COGON  C62  document  to  a  valid  Conemaugh  C62  document  and 
suggested  revisions 

-  Loaded  the  COGON  C62  document  into  the  CAL  emulator,  and  then  rendered  it  using  the 
CONNECT  Universal  Client  GUI. 
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•  On  July  1,  2013,  Allen  Barger  met  with  staff  members  from  Regenstrief  and  provided  them  with 
guidance  and  lessons  learned  regarding  C62  document  development. 

Transition  to  CONNECT  v3.3.1.3 

•  Evaluated  versions  of  OpenDS  that  are  suitable  for  deployment  with  the  Java  install  that  is 
required  by  CONNECT  v3.3. 1.3. 

•  Updated  the  MIDHT  code  base  to  work  with  the  CONNECT  v.3.3.1.3  Core  Libraries. 

•  Provided  TATRC  the  CONNECTUniversalClientGUI  binaries  for  use  with  CONNECT  v3.3.1.3 

•  Created  and  deployed  an  updated  Initiate  adapter  for  use  with  CONNECT  v3.3.  The  updated 
adapter  eliminates  an  extra  web  service  call  and  facilitates  the  use  of  the  CONNECT  build  script 
to  compile  the  adapter's  code. 

•  Updated  the  AdpaterCommonDataLayerEJB,  AdapterDocumentAssemblyProxyEJB, 
DocumentAssemblyManagerEJB,  DocumentManagerEJB,  NHINAdapterServicesEJB,  and 
TemplateManagerEJB  projects  to  accommodate  the  CONNECTCoreLib  changes  in  CONNECT 
V3.3.1.3. 

•  Resolved  the  CONNECT  Universal  Client  GUI  issue  that  caused  patient  id's  to  be  persisted  across 
GUI  sessions  and  included  in  the  SAML  header  of  future  transactions. 

•  Sample  C32  and  C62  documents  were  produced  from  the  transitioned  MIDHT  code  and  sent  to 
Conemaugh  for  review. 

•  The  CONNECT  v3.3.1.3  Entity  Doc  Query,  Adapter  Doc  Query,  and  NwHIN  Doc  Query  requests 
and  responses  were  compared  to  their  CONNECT  v3.1  counterparts. 

•  The  CONNECT  v3.3.1.3  C32  and  C62  documents  were  compared  to  their  CONNECT  v3.1 
counterparts. 

•  The  CONNECT  v3.3.1.3  document  repository,  audit  repository,  and  patient  correlation  tables 
were  compared  to  their  CONNECT  v3.1  counterparts. 

•  A  CONNECT  bug  may  be  responsible  for  wrongly  populating  the  Issuer  and  Issuer  Format  SAML 
Assertion  values  in  the  Audit  Log  and  Adapter  Document  Query  web  service  requests. 

•  Work  was  completed  on  creating  a  document  that  outlines  how  to  install  and  configure  the 
MIDHT  code  stack,  built  from  CONNECT  v3.3.1.3,  on  a  Windows-based  server. 

•  The  following  changes  were  made  to  the  MIDHT  CONNECT  v3.3.1.3  code  base: 

•  Properly  formatted  the  Date  of  Birth  (DOB)  and  telephone  number  values  in  the  Patient 
Discovery  request 

•  Added  an  "XDSDocumentEntryType"  slot,  used  to  request  both  stable  and  on-demand 
documents  from  eHealth  Exchange  participants,  to  the  Document  Query  request 

•  Changed  the  value  of  the  "returnComposedObjects"  attribute,  located  in  the  Document  Query 
request,  from  "false"  to  "true" 

•  Changed  the  Issuer  SAML  header  value  to  be  compliant  with  the  NwHIN  Authorization 
Framework  and  Oasis  SAML  specifications. 

•  Delivered  a  document  to  Conemaugh  that  outlines  how  to  install  and  configure  the  MIDHT  code 
stack,  built  from  CONNECT  v3.3.1.3,  on  a  Windows-based  server. 

•  The  following  changes  were  made  to  the  MIDHT  CONNECT  v3.3.1.3  code  base: 

Set  the  Initiate  min  score  to  99  for  both  internal  and  external  patient  lookups 
Prevented  the  CONNECT  Universal  Client  GUI  from  caching  the  internal  Initiate 
min  score  after  a  patient  lookup  is  performed  in  the  GUI 
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For  every  MIDHT  document  that  is  saved  to  the  document  repository,  populate 
the  document's  AvailabilityStatus  repository  field  with  "Available"  instead  of 
"Active" 

For  DQ  requests  that  originate  from  the  CONNECT  Universal  Client  GUI,  include 
a  $XDSDocumentEntryType  slot  in  the  request  if  the  targeted  community 
supports  the  2011  eHealth  Exchange  specifications. 

•  Enabled  complete  builds  of  the  CONNECT  code  stack  by  updating  the  AdapterFrameworklntTest 
project  to  work  with  architectural  changes  that  were  made  to  the  Dynamic  Document 
Assembler  code 

•  Increased  the  Document  Query  web  service  Proxy  timeout  threshold. 

•  The  migration  of  the  CONNECT  v3.3.1.3  MIDHT  code  base  from  the  MIDHT-PREPROD33  server 
to  the  MIDHT-GOLIVE  server  was  completed  on  September  19,  2013. 

VA  Partner  and  eHealth  Exchange  Testing 

•  On  November  13,  2012,  VA  test  cases  were  executed  to  monitor  C32  changes  that  were  made  in 
support  of  VA's  Adapter  Maintenance  Release  5.0.1.  During  the  testing  process,  several 
Conemaugh  initiated  Patient  Discovery,  Document  Query  and  Document  Retrieve  requests  were 
sent  to  the  VA  test  system  and  the  returned  responses  were  examined  for  accuracy. 

•  Northrop  Grumman  examined  the  VA  C32  module  content,  which  was  produced  during  Adapter 
5.0.1  validation  testing,  for  accuracy  and  reported  the  findings  to  Conemaugh. 

•  The  DIL  certificates  were  successfully  retrieved  from  Aegis  and  installed  on  the  MIDHT- 
PREPROD33  server. 

•  On  July  31,  2013,  Allen  Barger  attended  the  eHealth  Exchange  Participant  Testing  Webinar. 

•  The  uddiConnectionlnfo_CONNECT33_41.xml  file,  which  is  used  to  direct  the  CHS  gateway  to 
the  DIL  endpoints,  was  successfully  installed  on  the  MIDHT-PREPROD33  server. 

•  At  the  request  of  the  VA,  Document  Query  tests  were  conducted  for  test  patient  NWHINONE  on 
July  8, 15,  and  17,  2013.  The  test  results  were  extracted  from  Conemaugh's  server  log  and  sent 
to  the  VA  for  review. 

•  Helped  determine  if  a  NullPointerException,  which  prevented  the  DIL  from  sending  requests  to 
the  Conemaugh  test  gateway,  was  originating  from  the  DIL  or  Conemaugh's  gateway. 

•  Northrop  Grumman  helped  Conemaugh  interpret  a  DIL  error  message  regarding  a  missing 
patient  id,  and  determined  that  a  non-breaking  space  appended  to  the  end  of  the  patient  id  was 
the  cause  of  the  error. 

•  After  eHealth  Exchange  validation  testing  concluded,  the  certificates,  domain.xml  file,  and 
uddiConnectionlnfo.xml  file  on  the  MIDHT-PREPROD33  server  were  configured  for  validation 
testing  with  the  VA. 

•  On  September  11-12,  2013,  Allen  Barger  participated  in  the  validation  testing  sessions  that  took 
place  between  Conemaugh  and  the  VA. 

Lab  Result  Integration 

•  Using  sample  messages  provided  by  Conemaugh,  NG  successfully  parsed  HL7  v2.2  Lab  Result 
messages  using  the  open-source  HL7  Application  Programming  Interface  (HAPI)  library  to  HLV 
v3. 

•  Assisted  Conemaugh  with  identifying  the  data  objects  that  comprise  a  HL7  v2.2  message. 
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•  Coding  efforts  have  produced  a  C32  document  containing  a  Lab  Results  section  that  is  populated 
with  static  Laboratory  Observation  data.  As  changes  are  made  to  the  Lab  Results  section  of  the 
C32  document,  the  document's  adherence  to  the  CDA/CCD-based  specifications  are  routinely 
validated  using  the  NIST  testing  web  site. 

•  The  Document  Assembler  code  was  successfully  modified  to  accommodate  the  parsing  and 
interpretation  of  messages  containing  multiple  Lab  Orders  and  multiple  Lab  Results  (for  a  given 
Lab  Order). 

•  The  CHS  style  sheet  was  augmented  with  a  Lab  Results  section,  which  contains  a  Date/Time 
field,  a  Result  Type  field,  a  Result  Value  field,  a  Reference  Range  field,  an  Interpretation  field,  a 
Result  Status  field,  and  a  Source  field. 

•  Code  was  implemented  to  allow  the  represented  organization  for  each  Lab  result  to  dynamically 
change  based  on  the  hospital  code  value  in  the  MSH  section  of  the  ORU_R01  message. 

2014 -CGI Federal,  Inc. 

1.  Preproduction  certificate  update  -  The  eHealth  Exchange  validation  certificate  in  the  pre- 
production  environment  expired  in  January;  worked  with  John  to  update  that  certificate. 

-  Unfortunately  the  standard  process  for  importing  the  signed  eHealth  Exchange  certificate  back 
into  the  originating  keystore  no  longer  works.  The  suspicion  is  that  some  process  or 
technology  on  the  entrust  site  has  silently  changed,  it  is  common  knowledge  that  some  Java 
Development  Kit  (JDK)  implementations  create  slightly  differently  formatted  PEM  and  DER 
encoded  which  the  Oracle  JDK  has  trouble  with. 

-  Unfortunately  it's  difficult  to  confirm  this  as  the  culprit  without  input  on  the  technology  used 
on  the  Entrust  side.  After  many  hours  looking  at  this  issue  and  with  folks  from  MiHIN  (facing  a 
similar  issue)  we  were  able  to  create  a  workaround. 

-  The  work  around  involves  converting  the  original  keystore  to  the  PKCS12  format  and  then 
importing  the  signed  certificate  with  another  tool  called  OpenSSL,  then  converting  the 
certificate  back  to  JKS. 

2.  Preproduction  certificate  resolution  -  There  was  a  request  by  Healtheway  to  Entrust  to  modify 
the  algorithm  for  the  eHealth  Exchange  security  certs.  Creating  an  interoperability  issue  between 
older  eHealth  Exchange  certificates  and  the  newer  ones.  There  was  a  significant  amount  of 
research  performed  and  working  with  multiple  organizations  besides  Conemaugh  to  isolate  and 
resolve  the  issue. 

-  Though  Healtheway  stands  by  their  decision  that  it  was  a  good  and  necessary  certificate 
upgrade,  Healtheway  representatives  have  finally  agreed  with  the  findings  of  the  support  team 
and  are  working  with  Entrust  for  a  better  process  when  making  these  types  of  upgrades  and 
transitions  in  the  future. 

-  As  part  of  responding  to  the  larger  impacts  of  CONNECT  implementers  in  the  eHealth  Exchange 
community  the  support  team  wrote  up  and  contributed  documentation  to  1)  Detail  how  to 
update  a  new  root  certificate  from  the  eHealth  Exchange  and  2)  Key  troubleshooting  steps  for 
chain  of  trust  issues. 

3.  Researched  and  developed  a  fix  for  an  issue  with  test  UDDI  results  issue  where  the  maximum 
number  of  entries  of  100  was  hardcoded  in  the  code.  Tested  and  deployed  the  configurable 
return  value  for  UDDI  results  into  the  Conemaugh  environments. 

4.  Attended  meetings  and  worked  on  an  approach  to  on-demand  document  improvements  for 
Conemaugh.  The  approach  will  involve  a  Document  Assembly  Manager  to  incorporate  the  three 
separate  lab  messages  into  an  aggregated  single  synchronous  message  and  pass  it  to  the 
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Common  Access  Layer. 

5.  Added  lab  capabilities  to  the  Document  Assembly  Manager,  leveraging  the  code  developed  for 

3.1  and  merging  it  with  minor  modifications  for  3. 3. 1.3. 

-  Built  and  deployed  Conemaugh  lab  reporting  modifications  from  MIDHT_PHASE3  repository 

-  Tested  lab  reporting  modifications  and  found  missing  functionality  added  in  CONNECT  3.3 
(because  MIDHT_PHASE3  was  based  on  CONNECT  3.1),  made  necessary  updates 

-  Merged  lab  reporting  modifications  into  CONNECT  3.3  and  built  and  deployed 

-  Testing/troubleshooting  is  on-going,  continuing  to  coordinate  with  Conemaugh  resources 

-  CMIDHT-6  Updated  CommonDataLayer  Adapter  for  querying  lab  results 

-  CMIDHT-7  Updated  Lab  Response  Builder  to  pull  information  from  correct  message  type 

-  CMIDHT-8  Updated  information  source  of  CDA  header  for  correct  hospital  information 

-  CMIDHT-9  Modified  LabResponseBuilder  and  retrieve  Lab  Results  for  Patient  NWHINTWO 
NWHINZZZTESTPATIENT 

-  Supported  securing  Conemaugh  related  services,  an  issue  with  client-auth  in  production; 
ultimate  fix  was  not  small  and  not  non-invasive 

-  Merged  code  received  from  Conemaugh  into  3. 3. 1.3  codebase 

-  Fixed  build  issues  due  to  integrated  code  in  local  3. 3. 1.3  to  proceed 

-  Wrote  unit  tests  to  validate  the  flow  of  LabModule  work 

-  Added  interface  to  call  LabModule  in  CAL 

-  Made  required  code  changes  in  the  WSDL  provided  to  the  CAL  Service 

-  Made  required  code  changes  to  accommodate  the  Schema  for  the  same  service 

-  Modified  the  code  to  get  the  Response  (Response  as  String)  as  per  modified  WSDL  and  Schema 

-  Cleanup  of  previous  contractor  work  to  accommodate  receiving  Response  of  type  ORF_R04 
(was  ORU_R01) 

-  Performed  additional  analysis  on  message  variations  between  ORF_R04  and  ORU_R01  to 
account  for  and  properly  read  the  new  message  type 

6.  Work  also  continued  on  the  C32  clinical  document  builder: 

-  CMIDHT-10  -  Researched  HL7  lab  results  and  introduced  filtering  logic  to  filter  out  specific  text 
to  provide  the  correct  display  of  results  and  remove  unwanted  information. 

-  CMIDHT-11  -  Corrected  values  to  meet  the  VA  style  sheet  expectations.  Concatenated 
multiple  OBXs  with  the  same  ID  and  mirrored  how  the  VA  was  handling  comments. 

-  CMIDHT-13  -  Researched  issues  and  introduced  additional  search  criteria  for  "Reference 
range"  and  "Units"  in  the  OBX  text  block.  The  new  logic  properly  filter  and  displays  the  results 
in  the  F  observation. 

-  CMIDHT-14  -  Researched  results  and  added  additional  filtering  logic  to  properly  support 
certain  orders  such  as  a  blood  bank  order  which  appears  as  multiple  OBXs. 

-  CMIDHT-15  -  Updated  the  adapter  to  change  the  lab  result  type  to  populate  in  the  display 
name  not  the  reference  value  to  comply  with  the  VA  style  sheet. 

-  CMIDHT-16  -  Continued  to  refine  the  adapter  to  comply  with  the  VA  style  sheet  by  adding 
more  logic  to  handle  variations  in  the  lab  result  messages,  updated  the  XSL  to  pull  result  type 
for  display  from  display  name. 

-  CMIDHT-18  -  Inconsistencies  and  display  issues  with  lab  results  in  VA  VistA. 

•  Performed  research  and  developed  two  options,  presented  these  to  Conemaugh  for 
review. 

•  Received  guidance  to  proceed  with  the  second  option  detailed  in  the  JIRA  ticket. 

•  Developed,  tested  and  implemented  the  solution,  was  verified  with  the  VA. 
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-  CMIDHT-20  -  Add  OBX  information  to  comment  field  in  C32  instead  of  individual  result. 

•  Researched  supporting  specifications  and  reviewed  current  coding  to  see  how  this 
would  mesh  with  current  customized  adapter  logic. 

•  Discussed  via  email  with  Conemaugh  received  approval  with  the  following  logic 
contained  in  the  JIRA  ticket  comments  (August  28,  2014). 

7.  Provided  additional  research  and  guidance  related  to  HL7  lab  messages  and  the  best  way  to 
handle  the  variances  and  discern  what  the  VA  might  be  expecting  based  on  the  labs  result 
messages  being  supplied 

8.  Completed  support  for  Conemaugh  testing  with  the  VA  (In  their  Production/  Live  data  testing)  o 
Closed  the  ticket  used  for  monitoring  and  tracking  the  testing  work  CMIDHT-12,  all  open  issues 
are  resolved,  lab  result  variances  are  accounted  for  and  style-sheet  inconsistencies  have  been 
corrected 

•  Worked  with  the  VA  team  to  resolve  an  endpoint  issue  with  the  their  testing,  final 
result  seemed  to  indicate  it  was  an  issue  on  the  VA  side  which  was  resolved  and 
testing  continued 

-  CMIDHT-27  -  Integrate  CONNECT  Lab  Module  Adapter  Implementation  in  Conemaugh 
Production  Server 

•  Installed  Adapter  Implementation  of  Lab  Module  on  Conemaugh  Production  server 
and  support  testing  in  Production 

•  Moved  necessary  jars  and  configuration  files  that  are  added/updated  as  part  of  Lab 
integration  in  Adapter  Implementation 

•  Supported  testing  on  Production  server 

-  CMIDHT-29-  Completed  endpoint  configuration  verification  for  the  Test  environment  ensuring 
it  will  utilize  2011  versions  for  VA  testing.  Verification  included  the  following  service  endpoints 
PD,  QD  and  RD. 

•  The  2010  endpoints  specified  in  internalConnectionlnfo.xml  would  override  the  2011 
endpoints  of  uddiConnectionlnfo.xml.  The  2010  endpoints  in 
internalConnectionlnfo.xml  were  removed  as  part  of  this  ticket;  hence  the  Test 
CONNECT  gateway  will  be  using  uddiConnectionlnfo.xml  for  lookup  pulling  in  the  2011 
endpoints  contained  in  the  Healtheway  Test  UDDI. 

9.  Coordinated  with  the  eHealth  Exchange  personnel  on  securing  and  validating  the  new  EHEX 
Entrust  Production  Certificate. 

2015  -  CGI  Federal,  Inc. 

1.  CMIDHT-31  -  In  order  to  support  the  VA's  January  11th  transition  to  2011  NwHIN  specifications 
in  production  the  productionlnternalConnectionlnfo  file  was  updated  to  remove  2010  endpoints 
to  support  this  move  by  the  VA. 

•  The  file  was  updated  and  testing  conducted  to  ensure  proper  function.  Also  verified 
in  coordination  with  Conemaugh  and  VA  teams. 

2.  CMIDHT-32  -  Begin  to  perform  the  research  to  support  the  VA's  move  to  supporting  C-CDA 
documents.  There  was  a  concern  the  current  stylesheet  will  not  support  the  change  in 
document  types. 

•  Some  initial  research  was  performed  but  for  efficiency  the  decision  was  made  to 
request  a  fully  populated  sample  from  the  VA  to  ensure  all  sections  are  covered. 

•  Conemaugh  plans  to  retrieve  CCDA  documents  from  VA  going  forward.  VA  will 
support  both  C32s  and  CCDA,  and  will  have  a  system  in  place  to  provide  documents 
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(types)  based  on  the  remote  community.  Style-sheet  needs  to  be  updated  to 
accommodate  the  VA  CCDA. 

•  Created  a  list  of  the  items  and  updates  the  style-sheet  needs  to  be  updated  with  in 
order  to  accommodate  the  VA  CCDA. 

•  Made  the  necessary  changes  required  to  support  the  style-sheet  to  display  C-CDA 
documents. 

•  Verified  with  Conemaugh  that  testing  will  need  to  wait  until  the  "PurposeOfUse"  is 
changed  back  once  testing  of  this  data  element  is  complete. 

•  Reverted  back  to  the  "purpose  of  use"  to  "treatment"  and  began  testing  the  C-CDA 
stylesheet  changes. 

•  Researched  and  resolved  an  issue  with  the  XSL  and  IE8,  issue  was  a  discrepancy  that 
not  all  browsers  were  being  applied. 

•  Began  making  sure  NWHINFIVE  and  VA  SQA3  are  ready  for  testing,  understanding 
the  VA  stylesheet  incremental  changes  or  improvements. 

•  Researched  why  not  all  sections  are  showing  in  the  new  style  sheet. 

•  Determined  along  with  Conemaugh  the  best  timing  and  rollout  strategy  for  new 
stylesheet. 

3.  CMIDHT-33  -  Conemaugh  is  an  "unknown"  sender  in  production  VAP.  Initial  research  indicated 
that  the  VA  was  looking  for  a  couple  of  the  assertion  values  (namely  Organization  Id  and 
Organization)  are  not  being  populated  in  requests  from  Conemaugh  to  VA.  Ideally  these  values 
are  sent  through  the  entity  interface  assertion  element. 

•  This  issue  was  caused  as  the  VA  upgraded  to  CONNECT  4.2. 2. 2  with  CXF  as  the  web 
services  server  and  Conemaugh  is  on  CONNECT  3. 3. 1.3  using  Metro  as  the  web 
services  server. 

•  The  SAML  Assertion  Organization  Id  is  prepended  with  "urn:oid:  "  in  the  test 
environment  and  successfully  tested  sending  a  PD,DQ  and  DR  for  the  "NWHINONE" 
patient  and  provided  log  snippets  to  illustrate  compliance. 

4.  CONNECT  Timeout  Settings  Confirmation  -  Verified  and  reported  back  that  the  CONNECT 
services  were  set  to  20  minutes. 

5.  Testing  Purpose  of  Use  Elements  with  the  VA  -  The  VA  requested  conducting  a  test  with  them 
against  their  test  environments  using  the  data  element  'emergency'  for  the  Purpose  of  Use. 

•  The  values  were  modified  for  purposeOfUse  code  and  display  Name  to  "Emergency" 
in  Test  environment.  Restarted  the  gateway  and  self-query  tests  were  performed 
and  verified  the  purposeOfUse  code  and  displayName. 

•  Conemaugh  was  notified  the  gateway  was  ready  for  testing  with  the  VA.  Tests  were 
run  with  the  VA  and  awaited  results. 

•  Once  the  tests  were  completed  the  test  environment  was  reset  and  the  gateway 
restarted  for  future  testing. 

6.  Bug  found  in  SAML  Header  -  A  bug  was  discovered  during  the  preparation  for  the  style-sheet 
testing  with  the  VA.  The  bug  is  a  minor  logical  error  in  a  fix  that  was  implemented  for  the  prefix 
in  the  SAML  header. 

•  The  issue  was  resolved  and  patch  was  installed  in  both  test  and  production. 

Required  a  brief  down  time  for  the  server  restart. 

7.  2011  versus  2010  Specification  Support-  Verified  and  explained  to  Conemaugh  how  the 
CONNECT  3.3  gateway  and  the  Conemaugh  installation  are  set  up  to  support  2011  specifications 
(which  the  VA  has  migrated  to)  as  well  as  multi-version  specification  support. 
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8.  CMIDHT-36  -  Conemaugh  work  this  month  focused  on  addressing  the  final  remaining  formatting 
and  discrepancy  issues  and  move  style  sheet  fixes  for  C-CDA  clinical  document  display  in 
Conemaugh  Production  environment. 

•  Worked  to  understand  some  issues  the  VA  was  having  with  their  updated  C-CDA 
style  sheet,  how  those  changes  impacted  Conemaugh  and  coordinate  the  timing  of 
the  updates  as  they  addressed  these  issues. 

•  After  migrating  updated  style  sheet  into  production,  researched  issues  found  in  the 
following  sections: 

o  Overall  -  formatting  of  the  date  fields 
o  Lab  Results 
o  Immunizations 
o  Plan  of  Care 

•  Determined  the  cause  and  level  of  effort  for  the  issues  and  found  some  of  the  issues 
would  require  larger  than  expected  work.  Discussed  and  provided  more  explanation 
and  details  for  Conemaugh  to  make  a  determination. 

•  Proceeded  to  address  some  of  the  issues  per  direction  from  Conemaugh,  during  this 
time  Conemaugh  also  received  and  updated  C-CDA  which  seemed  to  address  some 
of  the  discrepancies  originally  being  detached. 

•  Until  further  solidifying  of  the  style-sheet  and  C-CDA  from  the  VA,  no  additional 
work  was  done  without  guidance  from  Conemaugh. 

9.  CMIDHT-39  -  Conemaugh  noticed  a  Universal  Client  issue  in  production,  there  was  an  issue  with 
the  list  boxes  for  the  Patient  Discovery  and  Documents  tab. 

•  Since  this  was  production  issue  support  was  somewhat  limited  due  to  access  and 
potential  impacts  to  production.  Conemaugh  supplied  screen  shots  and  results 
from  analyzing  the  log  files. 

•  Performed  a  detailed  analysis  of  the  information  provided  and  determined  that  the 
UDDI  seemed  to  be  populated  incorrectly  and  could  see  business  entries  for 
different  organizations. 

•  This  was  a  similar  issue  seen  before  with  Glassfish  2.1.1  at  startup.  Request  was 
made  for  the  Production  server  to  be  re-started  to  see  if  this  would  clear  the 
suspected  GlassFish  issues  and  resolve  the  original  problem  seen  in  the  UC. 

•  After  the  Production  server  was  restarted  the  issue  resolved,  once  Conemaugh 
validated  this  outcome  the  ticket  was  closed. 

10.  CMIDHT-41  -  After  reviewing  the  audit  log  Conemaugh  noticed  no  activity  from  June  3-8,  2015 
and  June  11-25,  2015  but  saw  activity  on  June  26,  2015. 

•  After  researching  it  was  determined  that  on  the  Production  server  the  Glassfish 
application  server  was  running  out  of  memory  showing  there  were  0  bytes  free. 
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Appendix  I.  NwHIN  Conditional  Acceptance. 


Fagp  I  of 


John  Hargreaves 


From:  Yeager.  Mari  arm  (CS/ONC)  {CTRj  [Mariano  .Yeagan^hhsjgtw] 

Bem:  TueEday,  September  27,  2011  3:12  PM 

TO:  Joe  Dado;  John  Hargreaves 

Ce:  M  ichael  Matthews;  vijay.3haJi@nrtorgrmjp.eorn ;  vrotjerts@nalmnaJaheaith.org 

BbUfect:  Exchange  Conditional  Acceptance 

Dear  Joseph  and  John: 


On  September  2£,  2011,  the  Exchange  Coordinaling  Committee  re-viewed  the  validation  testing  raauilE  lor  Conemaugh  Heaith  System 
rConemaugh").  Baaed  upon  this  information,  we  are  pleased  to  notify  you  that  Conemaugh  has  been  conditional  fy  accepted  aE  a 
Particpanl  «in  the  Exchange. 

This  oondtional  acceptance  requires  that  Conemaugh  be  ready  to  begin  ecxchangng  data  in  production  UEingtha  validated  Eerviceswith 
anulher  Exchange  Particpant  no  later  than  January  24,  201 2.  This  ia  one  hundred  twenty  (120)  calendar  days  following  the  Coordinating 
Committee's  conditional  aoceptanoa  oF  your  validation  testing  results . 

As  the  Coordinating  Committee  Chair  and  on  behalf  of  the  PartbpentE  in  the  Exchange,  I  will  countersign  Conenajgh's  DURSA  Joinder 
Agreement,  to  lake  effect  on  the  dale  you  go  into  production  as  am  Exchange  Participant. 

Please  note  that  once  the  amended  DURSA  takes  effect,  Conemaugh  will  be  required  Id  sign  the  amended  version.  We  will  contact  you 
with  mere  delaila  as  the  approval  pmcaas  progresses. 

If,  For  any  reason.  Conemaugh  is  unable  to  go  into  production  as  an  Exchange  Participant  by  January  24, 2012,  please  notify  your 
assigned  QNG  On  Boarding  Team  representative  and  Eubmit  an  extension  request  to  tiro  Coordinating  Committee  at 
onc.exchargaini8 DH@hha.gov.  The  Coordinating  Committee  may  accept  or  deny  thiE  extension  request  in  accordance  with  its  operating 
policiea  and  procedures. 

Conamaugh's  Formal  acceptance  as  a  Participant  takes  affect  on  the  date  Cane  maugh's  system  is  operational  in  a  production 
environment,  able  to  exchange  data  with  other  Participants,  Conemaugh^  DURSA  Joinder  Agreement  is  fully  ex  ecuted,  and 
when  Conemaugh's  Digital  Credentials  are  issued  and  Conemaugh  is  addEKl  lo  the  Exchange  service  registry. 

We  do  ask  that  you  withhold  announcements  about  you  r  partidpati on  until  ycur  Participation  goes  into  effect. 

The  fDllcwing  outlines  next  steps: 

*  An  on  boarding  team  repress maWe  will  isaue  Conemaugh  its  Production  Digital  Credentials  once  the  outstanding  issue  is 
add-essed  and  verified  by  the  on  boarding  team. 

*  You  will  be  asked  to  provide  the  on  boarding  team  the  required  information  to  add  Conemaugh  to  tha  Exchange  service 
registry.  The  on  boarding  team  will  confirm  the!  the  information  supplied  is  accurate  by  tearing  the  information  prmrided. 

■  The  on  boarding  team  will  issue  Conameugh  Digital  Credentials  in  the  production  regiatiy.  At  this  point,  Conemaugh  becomes 
actuated  as  a  Participant  m  the  Ex  change,  enabling  other  Participants  to  identify  and  begin  exchanging  health  information  with 
Conemaugh. 

If  you  have  any  questions  regarding  this  process,  please  do  no!  hesitate  to  contact  any  of  the  individuals  carbon  copied  on  this  letter. 


Regards, 


Mariano  Yeager  (Coordinating  Committee  Secretary)  on  behaH  of: 
Michael  MatfhewE 

Chair,  Exchange  Goordinati  ng  Com  mrttee 
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Appendix  J.  Continuity  of  Care  Document  (C32). 


FfTimrs 

lie  https :  //2D8 . 5 1. 45 . 4Q/CONNECTUniversalClientGUI/faces/Page3 .  jsp?dodd  = 

=  103. 8. 9284320. 020. 3590. 75  ~  13 186 14392^^]  |  !  *+  X  |  Google 

IZEi 

File  Edit 

View  Favorites  Tools  Help 

Ok  4ft 

$0  SUMMARIZATION  OF  EPISODE  NOTE 

0J  -  ED 

-  Page  -  Tools  - 

Conemaug h  Health  System 
SUMMARIZATION  OF  EPISODE  NOTE 

Created  on  14-OCT-2Q11 

Patient  Demographics 


Concmaugh 
I  lealth  System 


PATIENT:  JAMES  HAMILTON 

AD □  RESS  800  TELEPHONE  CT 

HONOLULU,  HI  96801 

Home:  te 1 : 8OS-3O0 -2343 


MRN:  34-5172 


BIRTHDATE 

SEX 

MARITAL  STATUS 


Q2-JUL-1  980 

Male 

Married 


NAME  : 
CONTACT  TVPE  : 


Table  of  Contents 


BARBARA  BROWN 
Emergency  Contact 


RELATIONSHIP:  spouse 

phone:  Home:  tel:-<-1 : 8 03-3 00-201 1 


•  Problems 

•  Allergies  and  Adverse  Reactions 

•  Medications 


return  to  top 


NAME 

CODE 

ONSET  DATE 

Urinary  Tract  Infection 

sse.o 

04-J  UN-2008 

Mtmm-ai  rtouime  ni^iccy  «.na  rnysii^ai  .aquii 

VTO.Q 

rteccfaea 

AJIertpes  and  Adverse  Reactions 

return  to  too 

SUBSTANCE 

EVENT  TYPE 

ONSET  DATE 

REACTION 

Cainpazine  TABS 

□  rug  Allergy 

04-D  EC-1  STS 

Tremor 

Pen  ici  1 1  ins 

□  rug  Allergy 

15-FEB-199C 

Tremor 

Medications 

return  to  too 

NAM  E 

SIG 

STATUS 

ROUTE 

Rx,  DATE 

ORDERED  BV 

Su  Ifaimethoxasole- 

TMPDS  800-160  MG 

Oral  Tablet 

TAKE  1  TABLET  DAILV. 

completed 

Ora  1 

0 1 -JAN-200S 

Provider  Allscripts. 

LORazepam  1  MG 

Oral  Tablet 

TAKE  1  TABLET  DAILV  AS  DIRECTED. 

completed 

Oral 

01-JAN-20QS 

Provider  Al  1  scripts. 

Boniva  ISO  MG  Oral 

Tablet 

TAKE  1  TABLET  ONCE  MONTHLY. 

completed 

Oral 

22-SEP-2Q1  1 

John  Carter 

FLUoxetine  HCI  20 

TAKE  1  CAPSULE  DAILV. 

completed 

Oral 

C1-JAN-2O08 

Provider  Al  1  scripts 

MG  <Oral  Cs  l*= 


Electronically  generated  by  Conemaugh  Health  System  on  October  14.  2011  _ 
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Appendix  K.  Emergency  Room  Discharge  Summary  (C62). 


File  Edit  Goto  Favorites  Help 

&  m  Suggested  Sites  m  Web  Slice  Gallery  ▼  m  UniversalClientGUI  m  TestDecode  m  CONNECTUniversalClient... 


j]  MIDHT  Clinical  Viewer  0]  DevConnectGUI 


©  M  |  :  ®  0  f 2 1  ©  ®  1 79-2%  I  - 1 1  H  Q  |  D  B  |  0 


PATIENT  NAME:  SMITH,  ELIZABETH 
DATE  OF  VISIT:  02/10/2011 

CHIEF  COMPLAINT:  Cough  and  vomiting. 

HISTORY  OF  PRESENT  ILLNESS:  The  patient  is  an  25-year-old  with  a  history  of 
asthma  presents  to  the  ER  with  a  2  day  history  of  nonproductive  cough, 
generalized  malaise,  decreased  p.o.  intake,  has  had  a  few  bouts  of  mucus-like 
vomiting.  Child  denies  headaches.  There  has  been  no  significant  rhinorrhea, 
ear  discomfort  or  throat  discomfort.  He  has  been  swallowing  without 
difficulty  although  mildly  decreased  appetite.  He  denies  neck  pain.  There 
is  no  chest  discomfort.  No  shortness  of  breath,  however,  he  was  sent  home 
from  school  because  father  states  because,  "His  lungs  did  not  sound  good." 
Child  denies  abdominal  pain.  There  has  been  no  bowel  or  bladder  changes. 
States  he  is  maintaining  normal  urine  output.  No  other  sick  contacts.  No 
recent  travel .  He  is  out  of  his  albuterol  MDI . 

PAST  MEDICAL  HISTORY:  As  above. 

MEDICATIONS:  Per  the  ED  record  and  reviewed. 

ALLERGIES :  No  known  drug  allergies . 

SOCIAL  HISTORY:  Collage  age  child,  no  sick  contacts,  no  recent  travel.  No 
reported  smoke  exposure. 


REVIEW  OF  SYSTEMS:  As  per  HPI, 
and  negative. 


otherwise  at  least  12  systems  were  reviewed 


PHYSICAL  EXAM:  An  25-year-old  in  no  acute  distress.  Vital  signs: 

Temperature  36.7,  heart  rate  94,  respirations  18,  blood  pressure  107/77,  02 
sat  99%  on  room  air,  pain  0/10.  HEENT:  Normocephalic,  atraumatic.  Pupils 
are  equal  and  reactive  bilaterally.  Extraocular  movements  are  intact. 

Sclerae  clear  without  drainage.  TMs  are  clear  bilaterally.  Nares  pink 
without  drainage.  Oropharynx  is  moist.  Appears  well-hydrated,  prominent 
tonsils,  however,  there  is  no  exudate.  No  evidence  of  peritonsillar  abscess. 
Uvula  is  at  midline.  No  other  intraoral  lesions.  Neck  is  supple.  No 
adenopathy.  No  meningeal  signs.  Heart  is  regular  without  murmur.  Lungs: 
Mild  expiratory  rhonchi  bilaterally.  Abdomen:  Soft,  nontender. 

Nondistended.  No  masses.  Extremities:  No  clubbing,  cyanosis  or  edema. 

Skin:  Warm,  dry,  no  rashes.  No  lesions.  Neurologic  exam:  Patient  is 

awake,  alert  and  oriented  times  3,  moving  all  extremities.  Gait  s  normal. 

DIAGNOSTIC  DATA:  PA  and  lateral  chest  x-ray  shows  hyper inf lated  lungs 
consistent  with  his  underlying  asthma  exacerbation.  This  is  on  preliminary 
reading. 


NAME:  SMITH,  ELIZABETH 

MR#:  000470366  ACCOUNT#:  8618506 

PHYSICIAN:  JOSEPH  A  MOONEY  M.D. 


LOCATION:  ER 

CONEMAUGH  VALLEY  MEMORIAL  HOSPITAL 
JOHNSTOWN,  PA 
Emergency  Room 
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Appendix  L.  VLER  Joint  Patient  Letter. 


Conemaugh 

Health  System  janK!i , .  v.,, 


V  A  Motiiuil  (  enter 


Dear  Veteran, 

There  is  a  new  way  to  share  information  with  your  healthcare  provider?  In  southwestern 
Pennsylvania! 

The  Virtual  Lifetime  Electronic  Record  |VLER)  health  program  will  allow  yw  to  share  your 
electronic  health  information,  via  the  Nationwide  Health  Information  Network  |MwHlN|.  with 
your  providers  who  are  members  of  the  Conernaugh  Health  System  and  the  James  £  Van  2andi 
VA  Medical  Center  in  Altoona,  PA. 

We  Invite  you  to  sign  up  for  the  VLER  Health  program  by  competing  and  signing  the  enclosed 
forms.  After  you  Join,  your  health  care  providers  will  be  able  to  see  your  current  medical  issues, 
emergency  room  reports,  discharge  summaries,  medications,  allergies,  and  other  important 
information  in  your  medical  record  as  necessary  for  your  treatment  this  information  will  be 
used  to  Improve  the  care  vou  receive  from  your  providers,  and  avoid  duplicating  services, 
Access  to  your  Information  will  be  strictly  controlled  and  protected  using  data  security 
standards  commonly  used  in  the  hanking  industry 

Once  you  have  completed  arid  signed  the  forms  we  have  included,  please  return  them  in  the 
envelope  provided.  Signing  these  forms  means  that  you  are  agreeing  to  share  your  health 
information  among  health  tare  providers  in  the  NwHlN  Exchange  who  provide  your  medical 
treatment. 

tl  you  have  questions,  please  contact 

John  Hargreaves.  Conemaugh  Health  System,  at  (S14)  269-S277,  and/or 

Barbara  Babiak,  VLE«  Coordinator.  James  E,  Van  7andt  VA  Medical  Center,  at  (0l4|  943-3164 

Ext.  0022. 

We  are  proud  to  serve  Veterans  that  served  our  country! 


f//  *<  *-  Cl 


William  MilU 

Medical  Center  Director 

James  E-  Van  2andt  VA  Medical  Center 


Scott  Becker 

Chief  Executive  Officer 

Conemaugh  Health  System 


f/  you  decide  not  la  jot  ft,  do  rgfJtoflfl  four  teaith  record  wifl  not  sfionetf  mth  prhalt  bioHhcare  fanttUes  usmy 
th  is  program.  Vour  Choke  Wiff  ngf  often  the  care  you  from  your  providers,  your  future  tore  nf  the  VA  or 

yput  va  benefits.  if  yov  eldest  id  join  ne^v  onrf  terrer rimfjfl#  j^ur  mind,  please  coMett  rte  VA  Aefease  of 
information  Office  tit  fS14f  943-9164  txt.  S022 
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Appendix  M.  Conemaugh  Consent  Form. 


Conemaugh 
^Health  System 


Request  for  and  Consent  to  Release  Protected  Health 
Information  to  the  e Health  Exchange 


Privacy  Act  Information:  By  completing  this  form,  no  release  of  information  will  occur  other  than  that  stated  below.  The 
form  allows  for  the  release  of  information  in  accordance  with  the  Health  Insurance  Portability  and  Accountability  Act 
(HIPAA),  45  CFR  Parts  160/164  and  5  U.S.C.  552a.  Your  consent  to  release  the  information  on  this  form  is  voluntary.  If  the 
information  containing  the  last  four  digits  of  your  Social  Security  Number  (SSN)  is  not  accurate,  the  eHealth  Exchange  will 
be  unable  to  comply  with  the  request.  The  SSN  will  be  used  to  locate  records  for  release.  The  Conemaugh  Health  System 
(CHS)  may  not  condition  treatment  or  other  actions  on  signing  of  this  consent.  CHS  may  release  the  information  you  put 
on  this  form  as  allowed  by  law.  CHS  may  make  "'routine  use"  disclosure  of  the  information  as  stated  in  the  CHS  "Notice  of 
Privacy  Practices".  Failure  to  participate  will  not  have  any  affect  on  care  to  which  you  may  receive.  CHS  may  also  use  this 
information  on  this  form  to  identify  patients  and  their  records.  Other  purposes  involving  this  information  may  occur  as 
allowed  by  law. 


Name: 


Last  4  digits  of  SSISI: 
Date  of  Birth: _ 


Requestor  Name: 


Approved  eHealth  Exchange  Participants 


Information  Requested: 

Health  information  from  electronic  health  record  systems. 


I  permit  CHS  to  release  my  protected  health  information  [PHI}  for  treatment  purposes  to  those  communities  that  are 
participating  in  the  eHealth  Exchange.  As  covered  by  38  U.S.C,  7332,  this  information  may  consist  of: 

■  Diagnosis  of  Sickle  Cell  Anemia 

*  T rea  tme  nt  of  o  r  referra  I  for  d  ru  g  a  b  u  se 

*  Treatment  of  or  referral  for  alcohol  abuse 

*  Treatment  of  or  testing  for  HIV/AIDS 

This  consent  covers  conditions  that  I  may  have  upon  signing  of  the  consent.  It  also  covers  conditions  that  I  may  acquire  in 
the  future.  This  consent  will  remain  in  effect  for  five  years.  I  may  withdraw  this  consent,  in  writing,  at  any  time.  I 
understand  that  actions  may  already  have  been  taken  to  comply  with  it.  Written  withdrawal  is  effective  upon  receipt  by 
CHS.  Redisclosure  of  my  electronic  health  records  by  others  may  occur  without  my  further  written  consent.  Redisclosure 
by  others  may  also  no  longer  be  protected. 


CONSENT:  I  certify  that  this  request  has  been  made  freely  and  without  force.  The  information  given  above  is  accurate  and 
complete  to  the  best  of  my  knowledge. 


Patient  Signature 


Date 
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Appendix  N.  Health  Information  Exchange  FAQs  Flyer. 


Health  Information 


Make  your  electronic  medical  records  available  to  your  providers. 


Conemaugh  Health  System  is  pleased  to  introduce 
sharing  of  health  records  through  the  Nationwide 
Health  Information  Network  (NwHIN).  The  goal  of 
the  project  is  to  securely  exchange  electronic 
medical  records  and  patient  data  with  external 
organizations,  including  the  Altoona  VA  Medical 
Center,  to  provide  veterans  timely  and  accurate 
healthcare  services. 

How  Does  It  Benefit  Me? 

Veterans  may  receive  some  of  their  care  from  non- 
VA  care  providers.  Electronic  medical  record  sharing 
allows  additional  providers  to  access  secure, 
updated  health  information  to  provide  you  accurate 
and  timely  care.  Electronic  files  also  eliminate  the 
need  for  you  to  carry  your  confidential  medical 
records  between  providers. 

Is  my  information  secure? 

Conformance  testing  to  ensure  compliance  with 
messaging,  privacy  and  security  specifications  at 
Conemaugh  Health  System  was  completed  with  the 
Office  of  the  National  Coordinator  for  Health 
information  Technology,  part  of  the  Department  of 
Health  and  Human  Services.  The  information  is  sent 
securely  through  the  Nationwide  Health  Information 
Network.  Only  healthcare  providers  that  are 
participating  in  the  exchange  will  be  able  to  view 
your  records  with  the  strict  purpose  of  providing 
direct  patient  care. 

EXCELLENCE.  EVERY  PATIENT.  EVERY  TIME. 


With  over  23,000  Veterans!  I 
in  Cambria  and  Somerset  V  1 
counties,  healthcare  IPl  ' 
providers  in  our  region 
will  have  quicker  access  to  more 
health  information  by  accessing 
the  exchange. 
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CONEMAUGH  HEALTH  SYSTEM 


Shared  Information  includes: 

*  Continuity  of  Care  (meds,  problems,  allergies)  *  Emergency  Room  Reports  *  Lab  Results 

*  Radiology  Reports  *  Discharge  Summaries 

I  Only  See  One  Provider.  Should  1  still  Join? 

Unfortunately,  unforeseen  accidents  and  emergencies  occur.  When  they  do,  Conemaugh  Health  System 
services  may  be  the  closest  available.  Providing  additional  health  information  to  health  care  entities  in  our 
network  can  help  save  your  life  in  an  emergency. 

How  Do  I  Participate? 

Participation  in  this  program  is  voluntary  and  free.  Signing  up  is  fast  and  easy!  Request  a  form  from  your  primary 
care  provider  or  call  814-269-5277.  Mail  form  to  John  Hargreaves,  Conemaugh  East  Hills,  1450  Scalp  Avenue, 
Suite  120,  Johnstown,  PA  15904 

What  If  I  Decide  Not  To  Join? 

Participation  in  this  program  is  strictly  voluntary.  Your  decision  not  participate  will  not  impact  the  quality  care  you 
receive  through  the  Conemaugh  Health  System. 


—  Conemaugh 
.Health  System 
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Appendix  O.  MIDHT  Press  Release. 


Conemaugh  Health  System  one  of  firs!  hospitals  lo  join  Nationwide  Health  Information  Network  (NwHL.  Pa^e  I  of  3 


Conemaugh  Health  System  one  of  first  S^tctn 

hospitals  to  join  Nationwide  Health 
Information  Network  (NwHIN) 


Home  >  About  Us  >  Media  C 
Infer  mafon  Natooik  (NwHIN) 


Conner  >  Pross  Fteleasos>  Cenemaugh  Haaltih  System  one  ef  fist  hospitals  to  join  Nationwide  Health 


Posted:  2012-09-11 

The  Conemaugh  Health  System  is  the  first  Pennsylvania  non-government  health  care  system  and  one  of 
just  30  in  the  United  States  to  go  “LIVE"  on  the  Nationwide  Health  Information  Network  (NwHIN) 

Exchange.  The  NwHIN  Exchange  is  sponsored  by  the  federal  government’s  Office  of  the  National 
Coordinator  as  a  way  to  securely  share  health  information  over  the  Internet  -  aka  Health  Information 
Exchange. 

The  Conemaugh  Health  System,  comprised  of  Conemaugh  Memorial  Medical  Center,  Conemaugh  Miners 
Medical  Center,  Conemaugh  Meyersdale  Medical  Center,  and  the  Conemaugh  Physician  Group  has 
attained  NwHIN  Exchange  operational  status  as  part  of  the  Military  Interoperable  Digital  Hospital  Testbed 
(MIDHT),  a  research  and  development  pilot  project  designed  to  promote  secure  health  information  sharing 
between  government  and  private  sector  healthcare  systems. 

The  goal  of  the  project  at  Conemaugh  is  to  securely  exchange  electronic  medical  records  and  patient  data 
with  external  organizations,  including  the  James  E.  Van  Zandt  VA  Medical  Center  in  Altoona  through  the 
Virtual  Lifetime  Electronic  Record  {VLER)  pilot  program.  With  over  3 , 000  veterans  seeking  care  from  both 
organizations,  clinicians  will  have  on  line  access  to  more  health  information  by  accessing  the  “NwHIN 
Exchange."  Conemaugh  will  provide  continuity  of  care  records,  hospital  discharge  summaries  and  radiology 
reports  to  partners  in  order  to  improve  care  coordination,  enhance  decision  making  and  reduce  duplicate 
testing.  Shared  patients  will  be  contacted  via  postal  mail  in  the  coming  weeks  to  enroll  into  the  program. 

“We  are  exerted  to  participate  in  the  VLER  program,  the  first  in  the  state  of  Pennsylvania.  I  encourage 
veterans  to  complete  both  consent  forms  and  hope  providers  utilize  the  “Exchange11  to  improve  care  in  our 
region,"  said  John  S.  Hargreaves,  MIDHT  Project  Manager. 

Conformance  testing  to  ensure  compliance  with  messaging,  privacy  and  security  specifications  was 
completed  with  the  Office  of  the  National  Coordinator  for  Health  Information  Technology,  part  of  the 
Department  of  Health  and  Human  Services.  After  review.,  Conemaugh  was  accepted  as  a  Participant  in  the 
“NwHIN  Exchange". 

The  MIDHT  project  is  sponsored  by  the  U.S.  Army  Medical  Research  &  Materiel  Command's  (USAMRMC) 
Telemedicine  &  Advanced  Technology  Research  Center  (TATRC)  and  is  funded  by  the  Department  of 
Defense  through  Contract  #  WB1XWH- 10-2-01 80.  Northrop  Grumman  Corporation  is  a  subcontractor  to  this 


http:  ffwwWj.  conemaugh.org/liemplatE_prEssrelease.  aspx  ?idt  1 02SS 


]m&20\2 
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Appendix  P.  CDA  Inbound  Documentation  Instructions. 


1.  Inbound  message  is  displayed  in  CarePortal. 


fe  http://thcpoitaK)l:8000/  -  HorizonWP  Physician  Portal  -  Windows  Internet  Explorer 


jlL> 


1  *  * 

my  portal  |  |  suspend  |  |  help  |  |  logout  | 

)  i  Sum  Viewer  DocView  ■  eM 


|  <Select> 


Def  Census  Search  Results  o  Trans  Cardio  Sum  Viewer  DocView  eMAR  Orders  Med  Record  Home  Facesheet  CPOE  CPG  PMH  Medical  Reconciliation 


Patient:  |  DORSEY,  DEBBIE  DORKIS;  000911876  ;G6-5555-l  |  Encounter:  1 12/05/13; ;  DR  NOT  IN  FILE  (a) 


CD  (HD  CM  Demographics  JSR  Portlet 

last  refresh:  19:13  (S)  ©  © 

Pat  Name 

Sex  Birth  Date  Age 

Location 

MR 

Number 

Enc  ID/Acct 
Number 

Allergies 

Adm  Date  ^,stc 
Date 

Attending 
Diagnosis  * 

DORSEY, 

DEBBIE 

DORKIS 

F  01/01/60  54 

00:00  Y 

G6-5555- 

1 

000911876 

1701610 

morphine,  Sulfa  (Sulfonamide  Antibiotics),  Shellfish 
Containing  Products,  latex,  egg,  iodine,  Grapefruit, 
pollen  extracts 

12/05/13 

15:58 

DR  NOT  IN 
FILE 

CD  (HD  CM  Transcription 


last  refresh:  19: 15  QD  (©  <5>) 


Report  for  DORSEY,  DEBBIE  DORKIS  (MRN:  000911876) 
Results  types  |ALL  Transcriptions  Result  test  classes  |  Edit  Preference  for  ALL  Transcriptions 

I*  Start  On:  |o 1/24/20 14  Go  Back:  [30  Days  T  New  reports  T  Today's  reports  T  Sort  reports  by  test  class 

Show  Selected  Reports  Show  All  Reports  |  item  search:  f 


Item  Search:  T 

Message  Subject 


Refresh 


From  Status  Hst 

diane.bengtson0748@direct.healthvault.com  Amended 


v®  Viewed  Date  Links  Description 

r  01/23/14  14:10  Exchange  Message  -  Inbound  CDA  Referral  for  New  Patient 

r  01/23/14  13:42  Exchange  Message  -  Inbound  CDA  Patient  information  for  Debbie  Dorkis  Dorsey  diane.bengtson0748@direct.healthvault.com  Amended 

Show  Selected  Reports  Show  All  Reports 


(Backto.Topj 


;son  HQRIZON 


2.  The  link  is  then  selected  to  view  the  message. 


Pabeot:  |  DORSFY,  nrnnir  dorkis;  nonet  i»76;  I  I?n7m;  null  ;r.6  ssss  I  *)  R1  |  Encounter.  |  *  >/os/l3;  ;  A;  on  not  in  fii  F;  ;  r.6  s.sss  i  («)  »| 


^55ff5T6mioyiyc>Jl>*drtl<t  i— tie*— *»■  nu 


Pot  Name 

Vox  Birth  unto  Age  Location 

MR 

Number 

rnr  in/Arrt 
Number 

Allergio* 

Admoato 

Attending 

DORSFY, 

DFBRIF 

DORKIS 

F  01/01/60  £•»  C6-S5SS- 

00.00  Y  1 

000011876 

1701610 

Sulfa  (Sulfonamide  Antibiotics),  Shellfish  Containing 
Products,  shelltish  derived,  morphine,  Stadol,  late., 
ludtrie,  Grapefruit.  pollen  extracts,  eyy 

12/05/13 

1S:SR 

DR  NOT  IN 
FUF 
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3.  The  "CCD  Inbound  CDA"  is  selected  to  open  the  attachment  to  view  the  patient 
record. 
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Appendix  Q.  CCDA  Outbound  Documentation  Instructions. 


1.  Login  to  Care  Manager. 

2.  Click  on  the  Report  Link. 

3.  Click  on  Document  Manager. 


File  Patient 
Care  Organizj: 
Patient:  TE! 
Age:  64  yr 
Diagnosis 


File  View 


All  Pats 


Qverdues 
0  To  Do  [40 


Patient  Name 


No  Changes 
For  TEST  C< 


Report  1  Review  Orders 


Document  Manager 


Admission  Discharge 
Med/Surg  Patient  Reports 
KRAMES 
Profile  Reports 

Lee  Ambulatory  Surgery  Reports 
Admin  Rx  Reports 
Crichton  Reports 
CR  Outpatient  Reports 
PMSlR  Reports 
RICN  Reports 
TCU  Reports 
WHC  Reports 

Orders  Department  Reports 
Dept  Meds/lVs  Reports 
Department  Reports 
heddetoxCCT 
Therapeutic  Scoring 
MedDCinst 
AdmMedRec 

DocViewer  HEO  Documents 
DocViewer  HMR  Documents 


]linical Apps  Tools  Window  Help  -^Logout 


rom: 

10/27/14  0  03:14  0 
0  El 


Q  Allergy  | 

|  Refresh  | 

Attending:  DR,  MOT 

Fac  -  Dept:  MMC-B6 

DOB:  05/05/1950 

Acct:  7211591 

Service:  HEM 

6507-2 

Admit  Dt: 

MRM:  00043609S 

ties  Expert  Orders  Help 

Assigned 

Patient 

Lab 

Rad 

HED 

Meds 

Transcription 

CPOE 

Exit 

ID/27/14  0  15:20  0 

0  0 


Dose/Order  Duration 


History 


Prty  Freq  (Rate) 


| Patient  Name  j 

|Patient  Location| 

|Med  Reconciliation  j 

|Facility  j 

|  Care  Team  , 

|MDTeam  J 

|Account#  | 

|mrn 

Itest,  COLLECTMGR 

B6  6507-2 

01 

AUX:  Brenda  P.  RN 

ATMD:  N.  Dr,CSMD:P.  Training 

7211591 

0004:1 

TEST.  MARIE  1 

IS  0400-1 

01 

9B76544 

0007< 

BOO  BOO.  BEAR 

IS  0400-2 

Needed  -  Admission  (9  weeks) 

01 

1357 

1357 

CHRISTEST.  HED 

IS  0401-1 

01 

RN:  Patricia T.  RN 

ATMD:  E.  Fikri  MD 

2342 

2342 

TEST.  SCHNAUZER 

IS  0401-2 

01 

ATMD:  A.  Smith  MO 

0009055 

0000' 

WONDERFUL  SUEM 

IS  0406-2 

01 

ATMD:  B.  Ahlstrom  MD 

0123 

304 

lONCALL.  RN 


"1  10/27/1415:14  I S Census  Patients  —  0  Meds  found.  0  IVs  found  0  Orders  found 
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4.  Click  on  the  Direct  Exchange  Link, 


File  Patient  Report  Review  Orders  Expert  Orders  HHS  Flowsheets  Clinical  Apps  Tools  Window  Help  ^Logout 

Care  Organizer  (Schedules)  Document  Manager  | 

Patient  TEST.  COLLECTMGR 

64  yr  Gender:  F 


DiaqiK 


I  Q Allergy  | 

Attending  DR.  NOT 
Service  '  HEM 


Fac 

Rm- 


MMC-B6 

6507-2 


05/05/1950 


Acct  721159 
MRN  00043 


Manage  Reports 
Manage  Requests 

Direct  Exchange 


Create  Direct  Exchange  Message 


Reports 

Date  &  Time 

Actioi 

B 

CDA  Patient  Information 

Current  Encounter 

View 

Send  selected  reports  via  :  Direct  Exchange... 


Previous  Direct  Exchange  Messages  (2) 

Created 

Sender 

Recipient 

Reports 

Status 

Actions 

04/07/2014  16:11 

BENGTSON,  DIANE 

diane.bengtson0748@direct.healthvault.com 

CDA  Patient  Information,  CDA  Patient 
Information 

SENT-HMS 

View  Details 

10/02/2014  15:03 

BENGTSON,  DIANE 

lisa  casey  @ch  i.allscriptsd  irect.  n  et 

CDA  Patient  Information,  CDA  Patient 
Information 

SENT-HMS 

View  Details 

5.  Click  the  checkbox  next  to  the  CCDA  Patient  Information  and  then  click  on  the  Direct 
Exchange  Tab  below  the  checkbox 


File  Patient  Report  Review  Orders  Expert  Orders  HHS  Flowsheets  Clinical  Apps  Tools  Window  Help  ^Logout 
Care  0rganizer{5chedules)  Document  Manager  | 

Patient:  TEST.  COLLECTMGR  Q  |  Q  Allergy  |  [  Refresh  | 

Ago:  64 yr  Gender  F  Attending:  DR,  NOT  Fac -Dept:  MMC-B6 

Diagnosis:  Service:  HEM  Rm-Bed:  6507-2 


Manage  Reports 
Manage  Requests 

Direct  Exchange 


Create  Direct  Exchange  Message 

Reports  Date  &  Time 

H]  CDA  Patient  Information  Current  Encounter 

Send  selected  reports  via :  Direct  Exchange... 


DOB:  05/05/1950  Acct:  7211591 

MRN:  00043609/ 


Actions 

View 


6.  Type  in  the  Direct  Exchange  Address  for  the  Recipient  To:  field 

a.  If  the  patient  is  being  referred  or  transitioned  to  PCRC  then  type: 
cpp(S)chi.allscriptsdirect.net 

b.  If  the  patient  is  being  referred  or  transitioned  to  Memorial  then  type: 
Memorial@direct.relavhealth.com 

c.  If  the  patient  is  being  referred  or  transitioned  to  Miners  then  type: 
Miners@direct.relavhealth.com 
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d.  If  the  patient  is  being  referred  or  transitioned  to  Meyersdale  then  type: 

Meversdale@direct.  relavhealth.com 

e.  If  the  patient  is  being  referred  or  transitioned  to  Home  Health  then  type: 
ConemaughHH@healthpoint.medicitv.net 

f.  If  the  patient  is  being  referred  or  transitioned  to  a  CPG  Provider  then  type: 
cpp(S)chi.allscriptsdirect.net 

g.  If  the  patient  is  requesting  a  personal  copy  of  their  CCDA  and  is  a  Follow  My  Health  Member 
then  type:  followmyhealth@fmh.allscriptsdirect.net 


File  Patient  Report  Review  Orders  Expert  Orders  HHS  Flowsheets  Clinical Apps  Tools  Window  Help  ^Logout 


Patient  TEST.  COLLECTMGR 

Age:  64  yr  Gender:  F 

Diagnosis: 

Manage  Reports 
Manage  Requests 

Direct  Exchange 

Attachments 


Care  QrganizerfSchedules)  Document  Manager 


M  |  Q  Allergy 


Refresh 


Attending:  DR.  NOT 
Service  HEM 


Fac  -  Dept:  MMC-B6 
Rm-Beti  6507-2 


DOB  05/05/1950 
Admit  Dt: 


Acct  7211591 
MRh  000436098 


Create  Direct  Exchange  Message 


Reports 

Date  &  Time 

Actions 

CD  A  Patient  Information 

Current  Encounter 

View 

Send  selected  reports  via  :  Direct  Exchange... 

Create  and  Send  Direct  Exchange  Message 
Patient  Name:  TEST,  COLLECTMGR  Patient  ID:  7211591 


I 


Selected  Patient  Reports  [1} 

Date  &  Time 

Status 

Action 

CDA  Patient  Information 

Current  Encounter 

Complete 

View  |  Remove 

You  will  need  to  type 

in  the  CPG  Physician 

tus 

Actions 

T-HMS 

View  Details 

T-HMS 

View  Details 

Recipients 


To:* 


^  Direct  Exch 

cpp@chLallscriptsdirect.net  J  ^  saati 

cpp@chi„ 


s  same  for  all  CPG  Physicians: 

♦allscriptsdirect.net 


Messaging 

Information 


O  Referral 
©  Transition 
Patient  Care 
Other 


^SeTectUie  Reason  the  patienFCCDATsTjemg 

I  sent?  i.e.  a  Referral?  Transition  of  Care? 
Patient  Care  Request  or  you  can  free  text 
the  another  reason  into  the  comment  field. 


Message: 


eSummary  of  Care  Document  for  Discharge  Follow  upr  Referrals  ^Transitions  in  Care. 
Patient  was  at  Memorial  from  10/01/14  to  09/30/15 


- 


Type  a  comment,  to  explain  why  the  patient  CCDA  is  being  sent. 


3959  chars  available 


BokJ*  =  Required. 


|  Send  I  Cancel  j 


7.  Click  the  corresponding  radio  button  for  the  Reason  the  CCDA  is  being  sent. 

a.  Referral  to  another  Service  Provider  such  as  PCRC  or  Home  Health 

b.  Transitioned  to  another  setting  of  care,  i.e.  Crichton,  CPG  Primary  Care  Provider,  etc. 

c.  Patient  Care- such  as  Hospice 
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d.  Other-  Requires  a  Reference  Reason  be  typed  in  corresponding  box 


8.  Type  the  rational  for  the  CCDA  being  sent  into  the  Message  Field  for  example: 

a.  eSummary  of  Care  Document  for  Referrals  and  Transitions  in  Care. 

Patient  was  at  Memorial  from  10/21/14  to  10/29/14 

b.  eSummary  of  Care  Document  for  Referrals  and  Transitions  in  Care. 

Patient  was  at  Miners  from  10/12/14  to  10/17/14 

c.  eSummary  of  Care  Document  for  Referrals  and  Transitions  in  Care. 

Patient  was  at  Meyersdale  from  10/14/14  to  10/16/14 

d.  eSummary  of  Care  Document  for  Admission  Referral  to  PCRC. 

Patient  was  admitted  at  Memorial  10/28/2014 

e.  eSummary  of  Care  Document  for  Referrals  and  Transitions  in  Care-Home  Health. 
Patient  was  at  Memorial  from  10/12/14  to  10/17/14 

9.  Click  the  send  tab. 
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Appendix  R.  Cross  tabulations  of  VLER  Patient  Survey 


Case  Processing  Summary 


Valid 

Cases 

Missing 

Total 

N 

Percent 

N 

Percent 

N 

Percent 

Did  you  feel  your  health  information  was  secure 
and  private?  *  Did  your  providers  talk  about  the 
new  service  during  your  appointments? 

112 

94.1% 

7 

5.9% 

119 

100.0% 

Gender:  *  Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

113 

95.0% 

6 

5.0% 

119 

100.0% 

Crosstab 


Did  your  providers  talk 
about  the  new  service 

during  your  appointments? 

YES 

NO 

Total 

YES 

Count 

42a 

39b 

81 

%  within  Did  you  feel  your  health  information  was 
secure  and  private? 

51.9% 

48.1% 

100.0% 

%  within  Did  your  providers  talk  about  the  new  service 

85.7% 

61.9% 

72.3% 

Did  you  feel 
your  health 
information 

during  your  appointments? 

%  of  Total 

Std.  Residual 

37.5% 

1.1 

34.8% 

-1.0 

72.3% 

NO  or 

Count 

7a 

24b 

31 

was  secure 

and  private? 

Not  Sure 

%  within  Did  you  feel  your  health  information  was 
secure  and  private? 

22.6% 

77.4% 

100.0% 

%  within  Did  your  providers  talk  about  the  new  service 
during  your  appointments? 

14.3% 

38.1% 

27.7% 

%  of  Total 

6.3% 

21.4% 

27.7% 

Std.  Residual 

-1.8 

1.6 

Total 

Count 

49 

63 

112 

%  within  Did  you  feel  your  health  information  was 
secure  and  private? 

43.8% 

56.3% 

100.0% 

%  within  Did  your  providers  talk  about  the  new  service 
during  your  appointments? 

100.0% 

100.0% 

100.0% 

%  of  Total 

43.8% 

56.3% 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories 
whose  column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 
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Chi-Square  Tests 


Value 

df  Asymp.  Sig.  (2-sided) 

Exact  Sig.  (2-sided)  Exact  Sig.  (1-sided) 

Pearson  Chi-Square 

7.806a 

1  .005 

Continuity  Correction13 

6.662 

1  .010 

Fisher's  Exact  Test 

.006  .004 

N  of  Valid  Cases 

112 

Risk  Estimate 

95%  Confidence  Interval 

Value  Lower  Upper 

Odds  Ratio  for  Did  you  feel  your  health  information  was  secure  and  private? 
(YES  /  NO  or  Not  Sure) 

3.692 

1.431 

9.529 

For  cohort  Did  your  providers  talk  about  the  new  service  during  your 
appointments?  =  YES 

2.296 

1.158 

4.554 

For  cohort  Did  your  providers  talk  about  the  new  service  during  your 
appointments?  =  NO 

.622 

.463 

.836 

N  of  Valid  Cases 

112 

a.  0  cells  (.0%)  have  expected  count  less  than  5.  The  minimum  expected  count  is  13.56. 

b.  Computed  only  for  a  2x2  table 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments?  Crosstabulation _ 


Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES 

NO 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Better  Coordination 
of  Care 

Count 

%  within  Please  select  which 
benefits  you  believe  were 
achieved  as  a  result  of  HIE 
between  Conemaugh  and 
Altoona  VA. 

41a 

52.6% 

37a 

47.4% 

78 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments?  Crosstabulation _ 


Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES 

NO 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Improved  Patient  Safety  Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

28a 

56.0% 

22a 

44.0% 

50 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 
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Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments?  Crosstabulation _ 

Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES  NO  Total 

Improved  Decision  Count  29a  30a  59 

Making/Quality  of  Care  o/o  withjn  P|ease  se|ect  49.2%  50.8%  100.0% 

which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 

_ and  Altoona  VA. _ 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 

Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments?  Crosstabulation _ 

Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES  NO  Total 

Less  Duplicate  Testing  Count  34a  33a  67 

%  within  Please  select  50.7%  49.3%  100.0% 

which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 

_ and  Altoona  VA. _ 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 


Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments?  Crosstabulation _ 


Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES 

NO 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Reduced  the  Need  to 
Hand  Carry  Records 

Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

32a 

48.5% 

34a 

51.5% 

66 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments?  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and 
Altoona  VA.  *  Did  your  providers  talk  about  the  new  service  during  your  appointments? 

Crosstabulation 


Did  your  providers  talk  about  the  new 
service  during  your  appointments? 

YES 

NO  Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

1  am  Not  Sure  Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

la 

25.0% 

3a  4 

75.0%  100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  your  providers  talk  about  the  new  service  during  your  appointments? 
categories  whose  column  proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 
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Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 


Did  you  feel  your  health  information  was 

secure  and  private? 

YES 

NO  or  Not  Sure 

Total 

Please  select  which 

Better  Coordination  of 

Count 

66a 

14a 

80 

benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Care 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

82.5% 

17.5% 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 


Did  you  feel  your  health  information  was 
secure  and  private? 

YES 

NO  or  Not  Sure 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Improved  Patient  Safety  Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

41a 

80.4% 

10a 

19.6% 

51 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 


Did  you  feel  your  health  information  was 
secure  and  private? 

YES 

NO  or  Not  Sure 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Improved  Decision 
Making/Quality  of  Care 

Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

49a 

80.3% 

12a 

19.7% 

61 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 


Did  you  feel  your  health  information  was 
secure  and  private? 

YES 

NO  or  Not  Sure 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Less  Duplicate  Testing  Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

50a 

74.6% 

17a 

25.4% 

67 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 
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Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 


Did  you  feel  your  health  information  was 
secure  and  private? 

YES 

NO  or  Not  Sure 

Total 

Please  select  which 
benefits  you  believe 
were  achieved  as  a 
result  of  HIE  between 
Conemaugh  and 
Altoona  VA. 

Reduced  the  Need  to 
Hand  Carry  Records 

Count 

%  within  Please  select 
which  benefits  you 
believe  were  achieved 
as  a  result  of  HIE 
between  Conemaugh 
and  Altoona  VA. 

51a 

76.1% 

16a 

23.9% 

67 

100.0% 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between  Conemaugh  and  Altoona 
_ VA.  *  Did  you  feel  your  health  information  was  secure  and  private?  Crosstabulation _ 

Did  you  feel  your  health  information  was 
_ secure  and  private? _ 

YES  NO  or  Not  Sure  Total 

I  am  Not  Sure  Count  2a  2a  4 

%  within  Please  select  50.0%  50.0%  100.0% 

which  benefits  you  believe 
were  achieved  as  a  result 
of  HIE  between 
Conemaugh  and  Altoona 

_ VA. _ 

Each  subscript  letter  denotes  a  subset  of  Did  you  feel  your  health  information  was  secure  and  private?  categories  whose  column 
proportions  do  not  differ  significantly  from  each  other  at  the  .05  level. 


Please  select  which 
benefits  you  believe  were 
achieved  as  a  result  of 
HIE  between  Conemaugh 
and  Altoona  VA. 
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